Jul 17 01:11:45.303200 ip-10-0-136-58 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 17 01:11:45.303431 ip-10-0-136-58 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 17 01:11:45.303517 ip-10-0-136-58 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 17 01:11:45.303930 ip-10-0-136-58 systemd[1]: Failed to start Kubernetes Kubelet. Jul 17 01:11:55.410664 ip-10-0-136-58 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 17 01:11:55.410686 ip-10-0-136-58 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a4547550a0974599b384335a9864411e -- Jul 17 01:13:53.503486 ip-10-0-136-58 systemd[1]: Starting Kubernetes Kubelet... Jul 17 01:13:53.890711 ip-10-0-136-58 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 01:13:53.890711 ip-10-0-136-58 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 17 01:13:53.890711 ip-10-0-136-58 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 01:13:53.890711 ip-10-0-136-58 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 17 01:13:53.890711 ip-10-0-136-58 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 01:13:53.892693 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.892548 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 17 01:13:53.897059 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897044 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 01:13:53.897059 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897060 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897064 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897067 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897070 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897073 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897075 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897078 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897081 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897083 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897087 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897089 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897092 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897094 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897100 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897103 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897106 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897108 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897112 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897114 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897116 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 01:13:53.897125 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897119 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897121 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897124 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897126 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897129 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897133 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897137 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897140 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897143 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897146 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897149 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897152 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897154 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897157 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897172 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897175 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897177 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897181 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897185 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 01:13:53.897619 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897188 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897190 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897194 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897196 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897198 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897201 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897203 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897206 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897209 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897211 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897213 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897216 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897218 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897220 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897223 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897226 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897229 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897231 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897234 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897237 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 01:13:53.898116 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897239 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897242 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897244 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897247 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897249 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897251 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897254 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897256 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897258 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897261 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897263 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897266 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897269 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897271 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897274 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897277 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897279 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897282 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897284 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897287 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 01:13:53.898625 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897289 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897292 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897296 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897299 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897301 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897304 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897306 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897310 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897312 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897314 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897317 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897319 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897322 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897325 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897327 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897330 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897332 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897334 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.897337 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898039 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 01:13:53.899099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898045 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898048 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898051 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898054 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898056 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898059 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898062 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898064 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898067 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898070 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898072 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898075 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898077 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898079 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898082 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898084 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898087 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898089 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898092 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898096 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 01:13:53.899604 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898098 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898101 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898104 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898106 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898109 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898111 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898114 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898116 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898119 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898121 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898124 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898127 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898129 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898132 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898134 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898136 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898139 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898143 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898145 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898148 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 01:13:53.900096 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898150 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898153 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898156 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898175 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898179 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898182 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898185 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898187 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898190 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898193 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898195 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898199 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898201 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898204 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898206 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898209 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898211 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898214 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898216 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 01:13:53.900630 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898219 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898221 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898224 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898226 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898228 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898231 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898234 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898236 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898239 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898243 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898246 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898249 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898252 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898254 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898257 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898261 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898263 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898267 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898269 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 01:13:53.901093 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898271 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898274 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898276 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898279 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898281 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898284 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898287 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898290 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898292 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898294 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898297 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898299 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898303 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898306 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898310 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898313 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898316 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898318 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898321 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 01:13:53.901568 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898323 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898724 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898729 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898731 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898734 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898736 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898740 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898742 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898745 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898748 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898750 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898753 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898755 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898758 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898761 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898763 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898766 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898768 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898771 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898774 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 01:13:53.902038 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898776 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898779 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898783 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898787 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898790 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898793 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898796 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898799 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898802 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898804 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898807 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898809 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898811 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898814 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898816 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898819 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898821 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898823 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898826 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 01:13:53.902536 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898828 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898831 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898833 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898836 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898838 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898841 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898843 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898846 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898848 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898850 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898853 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898858 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898860 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898863 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898865 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898868 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898870 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898873 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898875 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 01:13:53.903078 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898878 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898880 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898882 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898885 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898887 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898890 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898892 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898896 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898899 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898902 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898905 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898908 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898911 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898913 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898916 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898919 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898921 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898924 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898926 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898928 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 01:13:53.903562 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898931 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898933 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898936 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898938 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898943 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898946 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898948 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898950 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898953 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898955 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898958 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898962 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898965 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898967 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898970 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898972 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898975 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898977 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898979 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 01:13:53.904052 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898982 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898984 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.898986 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899059 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899066 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899074 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899081 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899088 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899092 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899099 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899105 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899110 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899113 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899117 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899120 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899123 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899126 2564 flags.go:64] FLAG: --cgroup-root="" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899130 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899133 2564 flags.go:64] FLAG: --client-ca-file="" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899135 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899138 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899142 2564 flags.go:64] FLAG: --cluster-domain="" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899145 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899148 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 17 01:13:53.904525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899151 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899154 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899170 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899174 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899177 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899180 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899183 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899186 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899189 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899194 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899197 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899202 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899204 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899207 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899210 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899213 2564 flags.go:64] FLAG: --enable-server="true" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899215 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899220 2564 flags.go:64] FLAG: --event-burst="100" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899223 2564 flags.go:64] FLAG: --event-qps="50" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899226 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899229 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899232 2564 flags.go:64] FLAG: --eviction-hard="" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899236 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899239 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899241 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 17 01:13:53.905108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899245 2564 flags.go:64] FLAG: --eviction-soft="" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899248 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899251 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899254 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899257 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899260 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899263 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899265 2564 flags.go:64] FLAG: --feature-gates="" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899269 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899272 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899275 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899278 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899280 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899283 2564 flags.go:64] FLAG: --help="false" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899286 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-136-58.ec2.internal" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899289 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899293 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899296 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899300 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899303 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899306 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899309 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899311 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 17 01:13:53.905743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899314 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899317 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899320 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899322 2564 flags.go:64] FLAG: --kube-reserved="" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899325 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899328 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899331 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899334 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899337 2564 flags.go:64] FLAG: --lock-file="" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899340 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899343 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899346 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899351 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899354 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899356 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899359 2564 flags.go:64] FLAG: --logging-format="text" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899362 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899365 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899367 2564 flags.go:64] FLAG: --manifest-url="" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899370 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899390 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899393 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899397 2564 flags.go:64] FLAG: --max-pods="110" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899401 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899404 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 17 01:13:53.906483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899408 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899411 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899414 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899416 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899419 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899426 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899429 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899432 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899435 2564 flags.go:64] FLAG: --pod-cidr="" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899437 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899443 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899446 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899449 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899452 2564 flags.go:64] FLAG: --port="10250" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899455 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899458 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0345fc7ee7775e0a2" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899461 2564 flags.go:64] FLAG: --qos-reserved="" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899466 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899468 2564 flags.go:64] FLAG: --register-node="true" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899471 2564 flags.go:64] FLAG: --register-with-taints="" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899475 2564 flags.go:64] FLAG: --registry-burst="10" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899478 2564 flags.go:64] FLAG: --registry-qps="5" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899481 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899483 2564 flags.go:64] FLAG: --reserved-memory="" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899487 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 17 01:13:53.907377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899490 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899493 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899496 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899498 2564 flags.go:64] FLAG: --runonce="false" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899501 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899504 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899507 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899511 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899514 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899517 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899520 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899523 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899525 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899528 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899530 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899533 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899536 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899539 2564 flags.go:64] FLAG: --system-cgroups="" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899541 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899546 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899548 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899560 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899565 2564 flags.go:64] FLAG: --tls-min-version="" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899568 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899572 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 17 01:13:53.908280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899574 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899577 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899580 2564 flags.go:64] FLAG: --v="2" Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899584 2564 flags.go:64] FLAG: --version="false" Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899588 2564 flags.go:64] FLAG: --vmodule="" Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899592 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.899595 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899710 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899714 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899717 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899720 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899723 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899726 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899729 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899733 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899736 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899739 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899742 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899745 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899749 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899752 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 01:13:53.908889 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899755 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899758 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899760 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899763 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899765 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899768 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899770 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899772 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899778 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899780 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899783 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899786 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899788 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899791 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899793 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899795 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899798 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899800 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899802 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899805 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 01:13:53.909412 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899807 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899809 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899812 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899814 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899817 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899820 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899823 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899825 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899828 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899830 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899832 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899834 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899837 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899840 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899843 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899846 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899849 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899851 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899854 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 01:13:53.909896 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899856 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899859 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899862 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899866 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899869 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899872 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899875 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899878 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899880 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899883 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899886 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899889 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899891 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899894 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899897 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899900 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899902 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899905 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899909 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899912 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 01:13:53.910401 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899915 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899918 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899920 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899923 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899926 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899929 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899931 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899934 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899936 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899939 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899941 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899944 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899946 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899949 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899951 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899954 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899957 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899959 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899961 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 01:13:53.910882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899964 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899966 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899969 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899971 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899973 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899976 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.899978 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.900420 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.908661 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.908677 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.908698 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.908703 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908762 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908768 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 01:13:53.911360 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908771 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908775 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908778 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908781 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908783 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908786 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908788 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908791 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908793 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908796 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908798 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908801 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908803 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908806 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908808 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908810 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908813 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908816 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908818 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908821 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 01:13:53.911710 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908823 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908826 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908829 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908831 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908834 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908836 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908840 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908843 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908846 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908848 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908851 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908854 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908856 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908858 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908861 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908863 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908866 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908868 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908871 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908873 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 01:13:53.912218 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908876 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908878 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908881 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908884 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908886 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908889 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908891 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908894 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908897 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908899 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908902 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908904 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908907 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908909 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908913 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908916 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908919 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908922 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908925 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 01:13:53.912708 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908928 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908931 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908934 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908937 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908940 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908942 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908945 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908949 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908952 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908955 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908958 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908960 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908962 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908965 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908967 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908970 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908972 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908975 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908978 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 01:13:53.913182 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908980 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908983 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908985 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908987 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908990 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908992 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908995 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.908997 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909000 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909002 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909005 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909007 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909010 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909013 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909015 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909018 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909020 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909023 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 01:13:53.913682 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909026 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.909031 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909143 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909148 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909151 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909154 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909157 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909182 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909185 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909187 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909190 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909193 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909195 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909198 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909201 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 01:13:53.914124 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909204 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909206 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909209 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909212 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909215 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909217 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909219 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909222 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909224 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909227 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909230 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909233 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909235 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909237 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909240 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909242 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909245 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909248 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909250 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909253 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 01:13:53.914528 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909255 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909258 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909260 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909263 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909265 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909268 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909270 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909272 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909275 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909277 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909280 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909282 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909285 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909287 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909289 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909292 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909294 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909296 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909299 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909301 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 01:13:53.915008 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909304 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909307 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909311 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909314 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909316 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909319 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909323 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909326 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909329 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909333 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909336 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909340 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909342 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909345 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909347 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909349 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909352 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909354 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 01:13:53.915508 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909356 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909359 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909362 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909364 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909367 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909369 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909372 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909375 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909378 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909380 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909383 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909385 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909388 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909390 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909392 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909395 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909397 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909400 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909402 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909405 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 01:13:53.915967 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909407 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909409 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909412 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909414 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909417 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909419 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909422 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:53.909424 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.909430 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.909547 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.912034 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.913136 2564 server.go:1019] "Starting client certificate rotation" Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.913201 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 01:13:53.916466 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.913240 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 01:13:53.916827 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.915838 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 01:13:53.918333 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.918316 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 01:13:53.929391 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.929367 2564 log.go:25] "Validated CRI v1 runtime API" Jul 17 01:13:53.931444 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.931422 2564 log.go:25] "Validated CRI v1 image API" Jul 17 01:13:53.932326 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.932311 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 17 01:13:53.938069 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.938040 2564 fs.go:135] Filesystem UUIDs: map[5eb1d166-267c-422f-9b6d-1291cd4e9be0:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f67e0dd4-0687-445d-9840-5c05bbc36efb:/dev/nvme0n1p4] Jul 17 01:13:53.938146 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.938067 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 17 01:13:53.943231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.943211 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 01:13:53.943655 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.943534 2564 manager.go:216] Machine: {Timestamp:2026-07-17 01:13:53.942374466 +0000 UTC m=+0.320298864 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099797 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e8f8b4778bb0a1a68e082ef3cd38c SystemUUID:ec2e8f8b-4778-bb0a-1a68-e082ef3cd38c BootID:a4547550-a097-4599-b384-335a9864411e Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:37:2f:59:69:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:37:2f:59:69:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:02:97:5b:34:06 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 17 01:13:53.943655 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.943651 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 17 01:13:53.943762 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.943741 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 17 01:13:53.946975 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.946790 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 17 01:13:53.947114 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.946978 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-58.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 17 01:13:53.947682 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.947670 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 17 01:13:53.947714 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.947685 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 17 01:13:53.947714 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.947692 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 01:13:53.948531 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.948520 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 01:13:53.949541 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.949531 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 17 01:13:53.952375 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.952363 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 17 01:13:53.952525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.952515 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 17 01:13:53.952557 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.952548 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 17 01:13:53.952587 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.952562 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 17 01:13:53.952587 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.952578 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 17 01:13:53.952587 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.952586 2564 kubelet.go:398] "Adding apiserver pod source" Jul 17 01:13:53.952685 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.952595 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 17 01:13:53.952980 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.952969 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 01:13:53.953028 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.952987 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 01:13:53.955234 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.955219 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 17 01:13:53.956806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.956792 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 17 01:13:53.956857 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.956823 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 17 01:13:53.957703 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957692 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 17 01:13:53.957736 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957708 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 17 01:13:53.957736 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957715 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 17 01:13:53.957736 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957721 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 17 01:13:53.957736 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957730 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 17 01:13:53.957736 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957735 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 17 01:13:53.957866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957743 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 17 01:13:53.957866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957748 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 17 01:13:53.957866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957754 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 17 01:13:53.957866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957759 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 17 01:13:53.957866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957768 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 17 01:13:53.957866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.957774 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 17 01:13:53.958968 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.958958 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 17 01:13:53.958968 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.958968 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 17 01:13:53.960955 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.960934 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vr85s" Jul 17 01:13:53.962142 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.962129 2564 server.go:1268] "Started kubelet" Jul 17 01:13:53.962386 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.962276 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 17 01:13:53.962386 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.962349 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 17 01:13:53.962545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.962282 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 17 01:13:53.962882 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:53.962842 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-58.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 17 01:13:53.962882 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:53.962845 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 17 01:13:53.963003 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.962936 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-58.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 17 01:13:53.963073 ip-10-0-136-58 systemd[1]: Started Kubernetes Kubelet. Jul 17 01:13:53.964500 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.964482 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 17 01:13:53.964997 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.964986 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 17 01:13:53.967225 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:53.967202 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 17 01:13:53.968141 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.968124 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vr85s" Jul 17 01:13:53.970924 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:53.970097 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-58.ec2.internal.18c2eeae1c816b0a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-58.ec2.internal,UID:ip-10-0-136-58.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-58.ec2.internal,},FirstTimestamp:2026-07-17 01:13:53.962101514 +0000 UTC m=+0.340025914,LastTimestamp:2026-07-17 01:13:53.962101514 +0000 UTC m=+0.340025914,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-58.ec2.internal,}" Jul 17 01:13:53.973303 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.973280 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 17 01:13:53.973387 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.973294 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 17 01:13:53.973725 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:53.973588 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:53.974404 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974219 2564 factory.go:55] Registering systemd factory Jul 17 01:13:53.974404 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974237 2564 factory.go:223] Registration of the systemd container factory successfully Jul 17 01:13:53.974542 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974455 2564 factory.go:153] Registering CRI-O factory Jul 17 01:13:53.974542 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974468 2564 factory.go:223] Registration of the crio container factory successfully Jul 17 01:13:53.974542 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974472 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 17 01:13:53.974542 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974519 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 17 01:13:53.974542 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974534 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 17 01:13:53.974542 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974521 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 17 01:13:53.974787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974563 2564 factory.go:103] Registering Raw factory Jul 17 01:13:53.974787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974573 2564 manager.go:1196] Started watching for new ooms in manager Jul 17 01:13:53.975205 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:53.975181 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-58.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 17 01:13:53.975303 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.975276 2564 manager.go:349] Starting recovery of all containers Jul 17 01:13:53.976148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.974678 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 17 01:13:53.976372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.976348 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 17 01:13:53.983414 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.983389 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 01:13:53.986033 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.985888 2564 manager.go:354] Recovery completed Jul 17 01:13:53.990403 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.990390 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 01:13:53.992290 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.992273 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientMemory" Jul 17 01:13:53.992371 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.992307 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 01:13:53.992371 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.992324 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientPID" Jul 17 01:13:53.992829 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.992816 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 17 01:13:53.992829 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.992826 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 01:13:53.992829 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.992828 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 17 01:13:53.992983 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.992846 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 17 01:13:53.993609 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.993592 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientMemory" Jul 17 01:13:53.993667 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.993623 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 01:13:53.993667 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.993634 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientPID" Jul 17 01:13:53.994051 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.994038 2564 policy_none.go:49] "None policy: Start" Jul 17 01:13:53.994108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.994057 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 17 01:13:53.994108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.994072 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 17 01:13:53.995004 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:53.994991 2564 policy_none.go:47] "Start" Jul 17 01:13:54.030914 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.030892 2564 manager.go:337] "Starting Device Plugin manager" Jul 17 01:13:54.031030 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.030928 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 17 01:13:54.031030 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.030941 2564 server.go:87] "Starting device plugin registration server" Jul 17 01:13:54.031275 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.031261 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 17 01:13:54.031311 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.031280 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 17 01:13:54.031393 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.031376 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 17 01:13:54.031480 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.031455 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 17 01:13:54.031480 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.031468 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 17 01:13:54.031983 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.031965 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 17 01:13:54.032042 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.032010 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.125707 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.125667 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 17 01:13:54.127043 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.127025 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 17 01:13:54.127143 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.127065 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 17 01:13:54.127143 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.127088 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 17 01:13:54.127143 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.127123 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 17 01:13:54.129055 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.129024 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 01:13:54.132153 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.132137 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 01:13:54.132869 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.132856 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientMemory" Jul 17 01:13:54.132937 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.132885 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 01:13:54.132937 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.132899 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientPID" Jul 17 01:13:54.132937 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.132920 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.139838 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.139823 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.139899 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.139844 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-58.ec2.internal\": node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.152648 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.152580 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.227639 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.227603 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal"] Jul 17 01:13:54.227825 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.227688 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 01:13:54.229564 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.229543 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientMemory" Jul 17 01:13:54.229672 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.229578 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 01:13:54.229672 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.229591 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientPID" Jul 17 01:13:54.230901 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.230884 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 01:13:54.231012 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.230985 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.231012 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.231009 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 01:13:54.232288 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.232271 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientMemory" Jul 17 01:13:54.232373 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.232295 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientMemory" Jul 17 01:13:54.232373 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.232299 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 01:13:54.232373 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.232313 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 01:13:54.232373 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.232322 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientPID" Jul 17 01:13:54.232373 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.232326 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientPID" Jul 17 01:13:54.233256 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.233243 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.233306 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.233265 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 01:13:54.233986 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.233973 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientMemory" Jul 17 01:13:54.234055 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.233995 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 01:13:54.234055 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.234004 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeHasSufficientPID" Jul 17 01:13:54.253199 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.253177 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.254883 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.254868 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-58.ec2.internal\" not found" node="ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.259397 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.259383 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-58.ec2.internal\" not found" node="ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.278512 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.278481 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e498e2dbaf2e1c392ffcb629d838109-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal\" (UID: \"7e498e2dbaf2e1c392ffcb629d838109\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.278512 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.278511 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e498e2dbaf2e1c392ffcb629d838109-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal\" (UID: \"7e498e2dbaf2e1c392ffcb629d838109\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.278681 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.278531 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/584482ea035b563238d63a3c04b310b9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-58.ec2.internal\" (UID: \"584482ea035b563238d63a3c04b310b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.353554 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.353510 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.378892 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.378864 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e498e2dbaf2e1c392ffcb629d838109-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal\" (UID: \"7e498e2dbaf2e1c392ffcb629d838109\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.378981 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.378908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7e498e2dbaf2e1c392ffcb629d838109-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal\" (UID: \"7e498e2dbaf2e1c392ffcb629d838109\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.378981 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.378966 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e498e2dbaf2e1c392ffcb629d838109-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal\" (UID: \"7e498e2dbaf2e1c392ffcb629d838109\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.379077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.378997 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/584482ea035b563238d63a3c04b310b9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-58.ec2.internal\" (UID: \"584482ea035b563238d63a3c04b310b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.379077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.379034 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/584482ea035b563238d63a3c04b310b9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-58.ec2.internal\" (UID: \"584482ea035b563238d63a3c04b310b9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.379077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.379053 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7e498e2dbaf2e1c392ffcb629d838109-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal\" (UID: \"7e498e2dbaf2e1c392ffcb629d838109\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.454156 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.454085 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.554663 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.554628 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.557821 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.557802 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.561714 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.561605 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal" Jul 17 01:13:54.654915 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.654877 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.755520 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.755440 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.855984 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.855947 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.913791 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.913749 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 17 01:13:54.914289 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.913955 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 01:13:54.914289 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.913955 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 01:13:54.956933 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:54.956898 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-58.ec2.internal\" not found" Jul 17 01:13:54.972465 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.972436 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 01:08:54 +0000 UTC" deadline="2028-01-30 04:15:02.332183149 +0000 UTC" Jul 17 01:13:54.972465 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:54.972465 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13491h1m7.359721551s" Jul 17 01:13:55.018374 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.018296 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 01:13:55.039490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.039456 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 01:13:55.074603 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.074574 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" Jul 17 01:13:55.092959 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.092933 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 01:13:55.094034 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.094021 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal" Jul 17 01:13:55.102496 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.102467 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 01:13:55.145636 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:55.145606 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e498e2dbaf2e1c392ffcb629d838109.slice/crio-66018828848be03a7b35cfd64b0e56fe9d19c1179b1c88d2a86c430f06b2ed7c WatchSource:0}: Error finding container 66018828848be03a7b35cfd64b0e56fe9d19c1179b1c88d2a86c430f06b2ed7c: Status 404 returned error can't find the container with id 66018828848be03a7b35cfd64b0e56fe9d19c1179b1c88d2a86c430f06b2ed7c Jul 17 01:13:55.153846 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.153826 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 01:13:55.161449 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:55.161426 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod584482ea035b563238d63a3c04b310b9.slice/crio-b2d96fc0f1f0502ec59068f4f2470f6f8501ab4415d1a900c67089286ee65b54 WatchSource:0}: Error finding container b2d96fc0f1f0502ec59068f4f2470f6f8501ab4415d1a900c67089286ee65b54: Status 404 returned error can't find the container with id b2d96fc0f1f0502ec59068f4f2470f6f8501ab4415d1a900c67089286ee65b54 Jul 17 01:13:55.953956 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.953924 2564 apiserver.go:52] "Watching apiserver" Jul 17 01:13:55.962521 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.962484 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 17 01:13:55.962917 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.962893 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-z8pf7","kube-system/konnectivity-agent-l9qrz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal","openshift-multus/network-metrics-daemon-qjf6m","openshift-network-diagnostics/network-check-target-64dhp","openshift-network-operator/iptables-alerter-kjqjm","openshift-ovn-kubernetes/ovnkube-node-6b52l","kube-system/global-pull-secret-syncer-zgf6x","kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk","openshift-cluster-node-tuning-operator/tuned-285tz","openshift-dns/node-resolver-7hnr7","openshift-image-registry/node-ca-jpvtd","openshift-multus/multus-additional-cni-plugins-j5f6x"] Jul 17 01:13:55.964435 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.964385 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 01:13:55.966455 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.966002 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.973213 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.968713 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 17 01:13:55.973213 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.968959 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qxp8x\"" Jul 17 01:13:55.973213 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.969484 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 17 01:13:55.973213 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.969719 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 17 01:13:55.973213 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.969959 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 17 01:13:55.973213 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.970128 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 17 01:13:55.973213 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.970933 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 17 01:13:55.974513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.974217 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 17 01:13:55.974513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.974396 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:55.977109 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.976675 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:55.981627 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.980954 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:55.981627 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:55.981136 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:13:55.982602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982088 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 17 01:13:55.982602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982161 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ccd26\"" Jul 17 01:13:55.982602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982211 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 17 01:13:55.982602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982273 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 17 01:13:55.982602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982423 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 17 01:13:55.982602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982452 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 17 01:13:55.982602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982475 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 17 01:13:55.982602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982580 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 17 01:13:55.983056 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982673 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 17 01:13:55.983056 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.982827 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fng76\"" Jul 17 01:13:55.984694 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.984665 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:13:55.984876 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:55.984827 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:13:55.987643 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.986848 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988722 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-os-release\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-run-openvswitch\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988788 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44020bfd-3d70-4b54-8782-12b022f858bb-ovnkube-script-lib\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv97g\" (UniqueName: \"kubernetes.io/projected/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-kube-api-access-dv97g\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988840 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-run-netns\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988858 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-etc-openvswitch\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988879 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhqbx\" (UniqueName: \"kubernetes.io/projected/86d44f9e-b02e-40ec-863f-b528b36e5f68-kube-api-access-nhqbx\") pod \"node-ca-jpvtd\" (UID: \"86d44f9e-b02e-40ec-863f-b528b36e5f68\") " pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988909 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-cnibin\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-systemd-units\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.988998 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-slash\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989022 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-run-systemd\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989082 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-node-log\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989105 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-log-socket\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989129 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.992313 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989158 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/86d44f9e-b02e-40ec-863f-b528b36e5f68-serviceca\") pod \"node-ca-jpvtd\" (UID: \"86d44f9e-b02e-40ec-863f-b528b36e5f68\") " pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44020bfd-3d70-4b54-8782-12b022f858bb-ovnkube-config\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989211 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44020bfd-3d70-4b54-8782-12b022f858bb-ovn-node-metrics-cert\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989260 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlcth\" (UniqueName: \"kubernetes.io/projected/44020bfd-3d70-4b54-8782-12b022f858bb-kube-api-access-zlcth\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989286 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989312 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989336 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-kubelet\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989404 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-run-ovn-kubernetes\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-cni-bin\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989460 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-cni-netd\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/86d44f9e-b02e-40ec-863f-b528b36e5f68-host\") pod \"node-ca-jpvtd\" (UID: \"86d44f9e-b02e-40ec-863f-b528b36e5f68\") " pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989508 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-system-cni-dir\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989545 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989571 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-var-lib-openvswitch\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989596 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-run-ovn\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.989623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44020bfd-3d70-4b54-8782-12b022f858bb-env-overrides\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.990435 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.991017 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 17 01:13:55.993295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.991338 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5mc4j\"" Jul 17 01:13:55.994290 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.991603 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 17 01:13:55.995787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.994716 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 01:13:55.995787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.995257 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:55.995787 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:55.995323 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:13:55.995787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.995420 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8pf7" Jul 17 01:13:55.996410 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.996389 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:13:55.998729 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.998687 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:55.998729 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.998697 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bkbm8\"" Jul 17 01:13:55.998877 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.998802 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 17 01:13:55.999020 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.998986 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 17 01:13:55.999125 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.999058 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 17 01:13:55.999263 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:55.999182 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8g8dr\"" Jul 17 01:13:56.001410 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.000884 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.001410 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.001227 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gs4w4\"" Jul 17 01:13:56.001759 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.001736 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 17 01:13:56.001759 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.001749 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 17 01:13:56.001873 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.001748 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 17 01:13:56.003431 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.003410 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kkd9c\"" Jul 17 01:13:56.003516 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.003448 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.003698 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.003664 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 17 01:13:56.004087 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.004063 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 17 01:13:56.006222 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.006195 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qwh4t\"" Jul 17 01:13:56.006366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.006344 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 17 01:13:56.006431 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.006367 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 17 01:13:56.013981 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.013955 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8cm4w" Jul 17 01:13:56.021996 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.021971 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8cm4w" Jul 17 01:13:56.076088 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.076065 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 17 01:13:56.090305 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.090242 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-modprobe-d\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.090486 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.090343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-sys\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.090486 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.090403 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-cni-netd\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.090728 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.090708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dcgx\" (UniqueName: \"kubernetes.io/projected/078ede28-d315-4b2a-862c-daeed6abdb41-kube-api-access-2dcgx\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:56.091060 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091038 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-cni-netd\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.091148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091094 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j8kf\" (UniqueName: \"kubernetes.io/projected/5622d70d-5b2e-43f6-9982-b8a931543303-kube-api-access-8j8kf\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.091148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091123 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-run\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.091290 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-var-lib-kubelet\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.091346 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091305 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-var-lib-kubelet\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.091396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091364 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b498z\" (UniqueName: \"kubernetes.io/projected/d65398b0-bacb-4d38-b91b-87d7b21beb1f-kube-api-access-b498z\") pod \"iptables-alerter-kjqjm\" (UID: \"d65398b0-bacb-4d38-b91b-87d7b21beb1f\") " pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.091396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091388 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-os-release\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.091499 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091445 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5622d70d-5b2e-43f6-9982-b8a931543303-cni-binary-copy\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.091499 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091469 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-multus-conf-dir\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.091597 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091512 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-run-openvswitch\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.091597 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091541 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44020bfd-3d70-4b54-8782-12b022f858bb-ovnkube-script-lib\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.091597 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091565 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991a0470-8bff-4780-8a40-693299c05626-tmp-dir\") pod \"node-resolver-7hnr7\" (UID: \"991a0470-8bff-4780-8a40-693299c05626\") " pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.091597 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091578 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-run-openvswitch\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.091792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091592 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv97g\" (UniqueName: \"kubernetes.io/projected/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-kube-api-access-dv97g\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.091792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091629 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-run-netns\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.091792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091647 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-etc-openvswitch\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.091792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-system-cni-dir\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.091792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091693 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-var-lib-cni-multus\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.091792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091711 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-systemd-units\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.091792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-run-systemd\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.091792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-node-log\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.091792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091778 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlcth\" (UniqueName: \"kubernetes.io/projected/44020bfd-3d70-4b54-8782-12b022f858bb-kube-api-access-zlcth\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5602e00b-be85-4eca-adbc-e7852aed270a-konnectivity-ca\") pod \"konnectivity-agent-l9qrz\" (UID: \"5602e00b-be85-4eca-adbc-e7852aed270a\") " pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091830 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d65398b0-bacb-4d38-b91b-87d7b21beb1f-iptables-alerter-script\") pod \"iptables-alerter-kjqjm\" (UID: \"d65398b0-bacb-4d38-b91b-87d7b21beb1f\") " pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091846 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-run-netns\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44020bfd-3d70-4b54-8782-12b022f858bb-ovnkube-config\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091889 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44020bfd-3d70-4b54-8782-12b022f858bb-ovn-node-metrics-cert\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091906 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d65398b0-bacb-4d38-b91b-87d7b21beb1f-host-slash\") pod \"iptables-alerter-kjqjm\" (UID: \"d65398b0-bacb-4d38-b91b-87d7b21beb1f\") " pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-etc-kubernetes\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091937 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-socket-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091960 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-registration-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091978 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb7x7\" (UniqueName: \"kubernetes.io/projected/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-kube-api-access-xb7x7\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.091993 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-sysctl-conf\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092010 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092028 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092052 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-kubelet\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.092226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092074 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-run-ovn-kubernetes\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092090 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-cni-bin\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092119 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-systemd\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4wl5\" (UniqueName: \"kubernetes.io/projected/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-kube-api-access-k4wl5\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092181 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/991a0470-8bff-4780-8a40-693299c05626-hosts-file\") pod \"node-resolver-7hnr7\" (UID: \"991a0470-8bff-4780-8a40-693299c05626\") " pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092200 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/86d44f9e-b02e-40ec-863f-b528b36e5f68-host\") pod \"node-ca-jpvtd\" (UID: \"86d44f9e-b02e-40ec-863f-b528b36e5f68\") " pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092258 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092286 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092314 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-multus-cni-dir\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092317 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/44020bfd-3d70-4b54-8782-12b022f858bb-ovnkube-script-lib\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092341 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-tuned\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092386 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-tmp\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092445 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-run-netns\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092480 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-system-cni-dir\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092512 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092601 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-etc-openvswitch\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092636 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-systemd-units\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.093097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092856 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092909 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-var-lib-openvswitch\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092938 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-run-ovn\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092965 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44020bfd-3d70-4b54-8782-12b022f858bb-env-overrides\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.092994 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-multus-socket-dir-parent\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093024 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5622d70d-5b2e-43f6-9982-b8a931543303-multus-daemon-config\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-os-release\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-cnibin\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093138 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-sysconfig\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093190 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-kubernetes\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-lib-modules\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-host\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5f557eb-8315-4fb5-803c-9d0c7961cb86-kubelet-config\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093312 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-device-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093337 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-sysctl-d\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093365 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhqbx\" (UniqueName: \"kubernetes.io/projected/86d44f9e-b02e-40ec-863f-b528b36e5f68-kube-api-access-nhqbx\") pod \"node-ca-jpvtd\" (UID: \"86d44f9e-b02e-40ec-863f-b528b36e5f68\") " pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-run-k8s-cni-cncf-io\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.094002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-var-lib-cni-bin\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093455 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-hostroot\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093513 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf7l4\" (UniqueName: \"kubernetes.io/projected/991a0470-8bff-4780-8a40-693299c05626-kube-api-access-wf7l4\") pod \"node-resolver-7hnr7\" (UID: \"991a0470-8bff-4780-8a40-693299c05626\") " pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093542 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-cnibin\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093569 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-slash\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-log-socket\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093725 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093769 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5f557eb-8315-4fb5-803c-9d0c7961cb86-dbus\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093811 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5602e00b-be85-4eca-adbc-e7852aed270a-agent-certs\") pod \"konnectivity-agent-l9qrz\" (UID: \"5602e00b-be85-4eca-adbc-e7852aed270a\") " pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.093865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-run-multus-certs\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094007 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/86d44f9e-b02e-40ec-863f-b528b36e5f68-serviceca\") pod \"node-ca-jpvtd\" (UID: \"86d44f9e-b02e-40ec-863f-b528b36e5f68\") " pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094083 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-var-lib-openvswitch\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-etc-selinux\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094129 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-sys-fs\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094134 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-run-systemd\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.096700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094190 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-node-log\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094210 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-run-ovn\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094323 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-slash\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094415 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-log-socket\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094448 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-os-release\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094510 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-cnibin\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094823 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/44020bfd-3d70-4b54-8782-12b022f858bb-env-overrides\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094884 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/44020bfd-3d70-4b54-8782-12b022f858bb-ovnkube-config\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.094960 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-system-cni-dir\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.095058 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.095135 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-run-ovn-kubernetes\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.095233 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.095561 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/86d44f9e-b02e-40ec-863f-b528b36e5f68-serviceca\") pod \"node-ca-jpvtd\" (UID: \"86d44f9e-b02e-40ec-863f-b528b36e5f68\") " pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.095656 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-cni-binary-copy\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.095706 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-kubelet\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.095747 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/86d44f9e-b02e-40ec-863f-b528b36e5f68-host\") pod \"node-ca-jpvtd\" (UID: \"86d44f9e-b02e-40ec-863f-b528b36e5f68\") " pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.095786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/44020bfd-3d70-4b54-8782-12b022f858bb-host-cni-bin\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.097566 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.095931 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.102709 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.102607 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/44020bfd-3d70-4b54-8782-12b022f858bb-ovn-node-metrics-cert\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.106174 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.105839 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhqbx\" (UniqueName: \"kubernetes.io/projected/86d44f9e-b02e-40ec-863f-b528b36e5f68-kube-api-access-nhqbx\") pod \"node-ca-jpvtd\" (UID: \"86d44f9e-b02e-40ec-863f-b528b36e5f68\") " pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:56.106174 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.106124 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlcth\" (UniqueName: \"kubernetes.io/projected/44020bfd-3d70-4b54-8782-12b022f858bb-kube-api-access-zlcth\") pod \"ovnkube-node-6b52l\" (UID: \"44020bfd-3d70-4b54-8782-12b022f858bb\") " pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.108953 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.108924 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv97g\" (UniqueName: \"kubernetes.io/projected/d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb-kube-api-access-dv97g\") pod \"multus-additional-cni-plugins-j5f6x\" (UID: \"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb\") " pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.133050 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.132990 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" event={"ID":"7e498e2dbaf2e1c392ffcb629d838109","Type":"ContainerStarted","Data":"66018828848be03a7b35cfd64b0e56fe9d19c1179b1c88d2a86c430f06b2ed7c"} Jul 17 01:13:56.134769 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.134736 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal" event={"ID":"584482ea035b563238d63a3c04b310b9","Type":"ContainerStarted","Data":"b2d96fc0f1f0502ec59068f4f2470f6f8501ab4415d1a900c67089286ee65b54"} Jul 17 01:13:56.194969 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.194933 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-multus-cni-dir\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.195144 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.194980 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-tuned\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.195144 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195008 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-tmp\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.195144 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195051 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-multus-socket-dir-parent\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.195144 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195078 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5622d70d-5b2e-43f6-9982-b8a931543303-multus-daemon-config\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.195144 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195117 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-cnibin\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.195144 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195143 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-sysconfig\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195171 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-kubernetes\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195218 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-lib-modules\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195240 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-host\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195270 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5f557eb-8315-4fb5-803c-9d0c7961cb86-kubelet-config\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195294 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-device-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195315 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-sysctl-d\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195342 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-run-k8s-cni-cncf-io\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195364 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-var-lib-cni-bin\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195387 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-hostroot\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195440 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf7l4\" (UniqueName: \"kubernetes.io/projected/991a0470-8bff-4780-8a40-693299c05626-kube-api-access-wf7l4\") pod \"node-resolver-7hnr7\" (UID: \"991a0470-8bff-4780-8a40-693299c05626\") " pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195470 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5f557eb-8315-4fb5-803c-9d0c7961cb86-dbus\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195494 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5602e00b-be85-4eca-adbc-e7852aed270a-agent-certs\") pod \"konnectivity-agent-l9qrz\" (UID: \"5602e00b-be85-4eca-adbc-e7852aed270a\") " pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:13:56.195513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-run-multus-certs\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-etc-selinux\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195569 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-sys-fs\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195593 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-modprobe-d\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195615 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-sys\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195640 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dcgx\" (UniqueName: \"kubernetes.io/projected/078ede28-d315-4b2a-862c-daeed6abdb41-kube-api-access-2dcgx\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195679 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j8kf\" (UniqueName: \"kubernetes.io/projected/5622d70d-5b2e-43f6-9982-b8a931543303-kube-api-access-8j8kf\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195703 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-run\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195733 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-var-lib-kubelet\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195759 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-var-lib-kubelet\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195792 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b498z\" (UniqueName: \"kubernetes.io/projected/d65398b0-bacb-4d38-b91b-87d7b21beb1f-kube-api-access-b498z\") pod \"iptables-alerter-kjqjm\" (UID: \"d65398b0-bacb-4d38-b91b-87d7b21beb1f\") " pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195822 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-os-release\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195845 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5622d70d-5b2e-43f6-9982-b8a931543303-cni-binary-copy\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-multus-conf-dir\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991a0470-8bff-4780-8a40-693299c05626-tmp-dir\") pod \"node-resolver-7hnr7\" (UID: \"991a0470-8bff-4780-8a40-693299c05626\") " pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195927 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-system-cni-dir\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195953 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-var-lib-cni-multus\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.195983 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:56.196115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196008 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5602e00b-be85-4eca-adbc-e7852aed270a-konnectivity-ca\") pod \"konnectivity-agent-l9qrz\" (UID: \"5602e00b-be85-4eca-adbc-e7852aed270a\") " pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196033 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d65398b0-bacb-4d38-b91b-87d7b21beb1f-iptables-alerter-script\") pod \"iptables-alerter-kjqjm\" (UID: \"d65398b0-bacb-4d38-b91b-87d7b21beb1f\") " pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196061 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-run-netns\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d65398b0-bacb-4d38-b91b-87d7b21beb1f-host-slash\") pod \"iptables-alerter-kjqjm\" (UID: \"d65398b0-bacb-4d38-b91b-87d7b21beb1f\") " pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-etc-kubernetes\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196137 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-socket-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196161 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-registration-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196187 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7x7\" (UniqueName: \"kubernetes.io/projected/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-kube-api-access-xb7x7\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196212 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-sysctl-conf\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-systemd\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4wl5\" (UniqueName: \"kubernetes.io/projected/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-kube-api-access-k4wl5\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196317 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/991a0470-8bff-4780-8a40-693299c05626-hosts-file\") pod \"node-resolver-7hnr7\" (UID: \"991a0470-8bff-4780-8a40-693299c05626\") " pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196343 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.196421 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-sys\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.196505 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 01:13:56.196855 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.196592 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret podName:a5f557eb-8315-4fb5-803c-9d0c7961cb86 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:56.696552703 +0000 UTC m=+3.074477093 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret") pod "global-pull-secret-syncer-zgf6x" (UID: "a5f557eb-8315-4fb5-803c-9d0c7961cb86") : object "kube-system"/"original-pull-secret" not registered Jul 17 01:13:56.197491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.197032 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-run-netns\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.197491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.197035 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-sysctl-d\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.197491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.197094 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d65398b0-bacb-4d38-b91b-87d7b21beb1f-host-slash\") pod \"iptables-alerter-kjqjm\" (UID: \"d65398b0-bacb-4d38-b91b-87d7b21beb1f\") " pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.197491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.197144 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-etc-kubernetes\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.197491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.197277 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-socket-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.197491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.197336 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-registration-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.197491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.197344 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5622d70d-5b2e-43f6-9982-b8a931543303-cni-binary-copy\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.202367 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.202326 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-multus-conf-dir\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.202690 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.202671 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/991a0470-8bff-4780-8a40-693299c05626-tmp-dir\") pod \"node-resolver-7hnr7\" (UID: \"991a0470-8bff-4780-8a40-693299c05626\") " pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.202791 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.202775 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-system-cni-dir\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.202861 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.202845 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-var-lib-cni-multus\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.202974 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.202960 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:13:56.203045 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.203035 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs podName:078ede28-d315-4b2a-862c-daeed6abdb41 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:56.703006807 +0000 UTC m=+3.080931211 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs") pod "network-metrics-daemon-qjf6m" (UID: "078ede28-d315-4b2a-862c-daeed6abdb41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:13:56.203335 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203312 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-run-k8s-cni-cncf-io\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.203420 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203408 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-var-lib-cni-bin\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.203508 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203492 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-hostroot\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.203592 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203576 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.203660 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203606 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5602e00b-be85-4eca-adbc-e7852aed270a-konnectivity-ca\") pod \"konnectivity-agent-l9qrz\" (UID: \"5602e00b-be85-4eca-adbc-e7852aed270a\") " pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:13:56.203660 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203652 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d65398b0-bacb-4d38-b91b-87d7b21beb1f-iptables-alerter-script\") pod \"iptables-alerter-kjqjm\" (UID: \"d65398b0-bacb-4d38-b91b-87d7b21beb1f\") " pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.203778 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203710 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-run\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.203996 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203940 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-tmp\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.203996 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203971 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a5f557eb-8315-4fb5-803c-9d0c7961cb86-dbus\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:56.204148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.203975 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-var-lib-kubelet\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.204148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.204027 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-var-lib-kubelet\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.204148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.204112 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-host-run-multus-certs\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.204148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.204124 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-tuned\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.204468 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.204391 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-os-release\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.204533 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.204481 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-sys-fs\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.204627 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.204606 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-etc-selinux\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.204761 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.204744 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-modprobe-d\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.204835 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.204819 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-kubernetes\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.204907 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.204892 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-multus-socket-dir-parent\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.205523 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.205293 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5622d70d-5b2e-43f6-9982-b8a931543303-multus-daemon-config\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.205523 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.205339 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-host\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.205523 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.205379 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-cnibin\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.205523 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.205476 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-lib-modules\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.206490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.206063 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5622d70d-5b2e-43f6-9982-b8a931543303-multus-cni-dir\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.206490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.206140 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-sysconfig\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.206490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.206182 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a5f557eb-8315-4fb5-803c-9d0c7961cb86-kubelet-config\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:56.206490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.206364 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-device-dir\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.206490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.206380 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-sysctl-conf\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.206490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.206486 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-etc-systemd\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.207435 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.206597 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/991a0470-8bff-4780-8a40-693299c05626-hosts-file\") pod \"node-resolver-7hnr7\" (UID: \"991a0470-8bff-4780-8a40-693299c05626\") " pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.207875 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.207854 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5602e00b-be85-4eca-adbc-e7852aed270a-agent-certs\") pod \"konnectivity-agent-l9qrz\" (UID: \"5602e00b-be85-4eca-adbc-e7852aed270a\") " pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:13:56.208060 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.208041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dcgx\" (UniqueName: \"kubernetes.io/projected/078ede28-d315-4b2a-862c-daeed6abdb41-kube-api-access-2dcgx\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:56.217295 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.217271 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 01:13:56.217409 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.217299 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 01:13:56.217409 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.217315 2564 projected.go:196] Error preparing data for projected volume kube-api-access-j9nhm for pod openshift-network-diagnostics/network-check-target-64dhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:13:56.217409 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.217391 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm podName:d7ed04a1-66ad-456b-93b4-1ddf01000212 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:56.717370464 +0000 UTC m=+3.095294876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-j9nhm" (UniqueName: "kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm") pod "network-check-target-64dhp" (UID: "d7ed04a1-66ad-456b-93b4-1ddf01000212") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:13:56.217640 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.217620 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b498z\" (UniqueName: \"kubernetes.io/projected/d65398b0-bacb-4d38-b91b-87d7b21beb1f-kube-api-access-b498z\") pod \"iptables-alerter-kjqjm\" (UID: \"d65398b0-bacb-4d38-b91b-87d7b21beb1f\") " pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.217808 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.217785 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf7l4\" (UniqueName: \"kubernetes.io/projected/991a0470-8bff-4780-8a40-693299c05626-kube-api-access-wf7l4\") pod \"node-resolver-7hnr7\" (UID: \"991a0470-8bff-4780-8a40-693299c05626\") " pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.218479 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.218457 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb7x7\" (UniqueName: \"kubernetes.io/projected/85108d9c-6817-4c3c-bf3f-6fee0ebbe847-kube-api-access-xb7x7\") pod \"aws-ebs-csi-driver-node-9tjqk\" (UID: \"85108d9c-6817-4c3c-bf3f-6fee0ebbe847\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.218792 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.218772 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4wl5\" (UniqueName: \"kubernetes.io/projected/b5a2b823-b37e-4a93-8e0d-00b83715ec5f-kube-api-access-k4wl5\") pod \"tuned-285tz\" (UID: \"b5a2b823-b37e-4a93-8e0d-00b83715ec5f\") " pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.219421 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.219405 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j8kf\" (UniqueName: \"kubernetes.io/projected/5622d70d-5b2e-43f6-9982-b8a931543303-kube-api-access-8j8kf\") pod \"multus-z8pf7\" (UID: \"5622d70d-5b2e-43f6-9982-b8a931543303\") " pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.292423 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.292393 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:13:56.311808 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.311771 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jpvtd" Jul 17 01:13:56.320847 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.320822 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" Jul 17 01:13:56.324400 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.324381 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 01:13:56.328741 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.328716 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kjqjm" Jul 17 01:13:56.339937 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.339914 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z8pf7" Jul 17 01:13:56.348918 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.348893 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:13:56.362232 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.361991 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" Jul 17 01:13:56.366168 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.366147 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-285tz" Jul 17 01:13:56.372117 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.372096 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7hnr7" Jul 17 01:13:56.700604 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.700514 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:56.700751 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.700678 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 01:13:56.700751 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.700744 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret podName:a5f557eb-8315-4fb5-803c-9d0c7961cb86 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.700727535 +0000 UTC m=+4.078651941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret") pod "global-pull-secret-syncer-zgf6x" (UID: "a5f557eb-8315-4fb5-803c-9d0c7961cb86") : object "kube-system"/"original-pull-secret" not registered Jul 17 01:13:56.801290 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.801252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:56.801464 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:56.801325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:13:56.801464 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.801425 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:13:56.801589 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.801486 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 01:13:56.801589 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.801501 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 01:13:56.801589 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.801510 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs podName:078ede28-d315-4b2a-862c-daeed6abdb41 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.801485819 +0000 UTC m=+4.179410233 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs") pod "network-metrics-daemon-qjf6m" (UID: "078ede28-d315-4b2a-862c-daeed6abdb41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:13:56.801589 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.801512 2564 projected.go:196] Error preparing data for projected volume kube-api-access-j9nhm for pod openshift-network-diagnostics/network-check-target-64dhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:13:56.801589 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:56.801564 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm podName:d7ed04a1-66ad-456b-93b4-1ddf01000212 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:57.801550334 +0000 UTC m=+4.179474721 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9nhm" (UniqueName: "kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm") pod "network-check-target-64dhp" (UID: "d7ed04a1-66ad-456b-93b4-1ddf01000212") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:13:57.025540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:57.025410 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 01:08:56 +0000 UTC" deadline="2028-01-16 16:25:46.265653437 +0000 UTC" Jul 17 01:13:57.025540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:57.025472 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13167h11m49.240183833s" Jul 17 01:13:57.128373 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:57.128337 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:13:57.128555 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:57.128342 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:57.128555 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.128477 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:13:57.128662 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.128594 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:13:57.307152 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:57.306386 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44020bfd_3d70_4b54_8782_12b022f858bb.slice/crio-ad51c4eff1e3627203bc07c1db05a789f227a073f2b050d5f52df4092116fbdc WatchSource:0}: Error finding container ad51c4eff1e3627203bc07c1db05a789f227a073f2b050d5f52df4092116fbdc: Status 404 returned error can't find the container with id ad51c4eff1e3627203bc07c1db05a789f227a073f2b050d5f52df4092116fbdc Jul 17 01:13:57.308675 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:57.308647 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85108d9c_6817_4c3c_bf3f_6fee0ebbe847.slice/crio-5f2b9e3311606a4db3164792c5b03339d4d108476ba361877d9f9246fc9b34d9 WatchSource:0}: Error finding container 5f2b9e3311606a4db3164792c5b03339d4d108476ba361877d9f9246fc9b34d9: Status 404 returned error can't find the container with id 5f2b9e3311606a4db3164792c5b03339d4d108476ba361877d9f9246fc9b34d9 Jul 17 01:13:57.310756 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:57.310677 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d44f9e_b02e_40ec_863f_b528b36e5f68.slice/crio-1f41e62e8ff6e62cc8d95d953ac8c86bf3d3e2b258434408e50d28902e64eb00 WatchSource:0}: Error finding container 1f41e62e8ff6e62cc8d95d953ac8c86bf3d3e2b258434408e50d28902e64eb00: Status 404 returned error can't find the container with id 1f41e62e8ff6e62cc8d95d953ac8c86bf3d3e2b258434408e50d28902e64eb00 Jul 17 01:13:57.312664 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:57.312639 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd65398b0_bacb_4d38_b91b_87d7b21beb1f.slice/crio-80d71893ea0d04debefa9598d37c911f57c143f0b6b69a20df77a90f8e5f253a WatchSource:0}: Error finding container 80d71893ea0d04debefa9598d37c911f57c143f0b6b69a20df77a90f8e5f253a: Status 404 returned error can't find the container with id 80d71893ea0d04debefa9598d37c911f57c143f0b6b69a20df77a90f8e5f253a Jul 17 01:13:57.313651 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:57.313616 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991a0470_8bff_4780_8a40_693299c05626.slice/crio-3c1cff96f21960fae7b7760f68947149ac27b0c18ea6f971a7a677109134e74d WatchSource:0}: Error finding container 3c1cff96f21960fae7b7760f68947149ac27b0c18ea6f971a7a677109134e74d: Status 404 returned error can't find the container with id 3c1cff96f21960fae7b7760f68947149ac27b0c18ea6f971a7a677109134e74d Jul 17 01:13:57.316352 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:57.316020 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5602e00b_be85_4eca_adbc_e7852aed270a.slice/crio-2702c5ef67b80b40e1ff7b5248a7771184f94e475d207c614cce74bfaf3a46cb WatchSource:0}: Error finding container 2702c5ef67b80b40e1ff7b5248a7771184f94e475d207c614cce74bfaf3a46cb: Status 404 returned error can't find the container with id 2702c5ef67b80b40e1ff7b5248a7771184f94e475d207c614cce74bfaf3a46cb Jul 17 01:13:57.321393 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:13:57.321368 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5a5c7c0_fa10_4f3c_b66e_c329c0b65cbb.slice/crio-859d58ca2406d6392eacf737601f3469431af22e25064ba0bdf3d3e442de7919 WatchSource:0}: Error finding container 859d58ca2406d6392eacf737601f3469431af22e25064ba0bdf3d3e442de7919: Status 404 returned error can't find the container with id 859d58ca2406d6392eacf737601f3469431af22e25064ba0bdf3d3e442de7919 Jul 17 01:13:57.708415 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:57.708164 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:57.708560 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.708343 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 01:13:57.708560 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.708517 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret podName:a5f557eb-8315-4fb5-803c-9d0c7961cb86 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:59.708499324 +0000 UTC m=+6.086423710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret") pod "global-pull-secret-syncer-zgf6x" (UID: "a5f557eb-8315-4fb5-803c-9d0c7961cb86") : object "kube-system"/"original-pull-secret" not registered Jul 17 01:13:57.809579 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:57.809490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:57.809579 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:57.809558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:13:57.809810 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.809641 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:13:57.809810 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.809720 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs podName:078ede28-d315-4b2a-862c-daeed6abdb41 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:59.809696494 +0000 UTC m=+6.187620894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs") pod "network-metrics-daemon-qjf6m" (UID: "078ede28-d315-4b2a-862c-daeed6abdb41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:13:57.809810 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.809734 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 01:13:57.809810 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.809763 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 01:13:57.809810 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.809777 2564 projected.go:196] Error preparing data for projected volume kube-api-access-j9nhm for pod openshift-network-diagnostics/network-check-target-64dhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:13:57.810084 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:57.809828 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm podName:d7ed04a1-66ad-456b-93b4-1ddf01000212 nodeName:}" failed. No retries permitted until 2026-07-17 01:13:59.809812333 +0000 UTC m=+6.187736724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9nhm" (UniqueName: "kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm") pod "network-check-target-64dhp" (UID: "d7ed04a1-66ad-456b-93b4-1ddf01000212") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:13:58.026110 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.026059 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 01:08:56 +0000 UTC" deadline="2027-12-29 08:19:39.018068733 +0000 UTC" Jul 17 01:13:58.026110 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.026108 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12727h5m40.991965453s" Jul 17 01:13:58.138200 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.138115 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:58.138343 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:58.138313 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:13:58.147187 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.147139 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jpvtd" event={"ID":"86d44f9e-b02e-40ec-863f-b528b36e5f68","Type":"ContainerStarted","Data":"1f41e62e8ff6e62cc8d95d953ac8c86bf3d3e2b258434408e50d28902e64eb00"} Jul 17 01:13:58.152697 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.152662 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" event={"ID":"85108d9c-6817-4c3c-bf3f-6fee0ebbe847","Type":"ContainerStarted","Data":"5f2b9e3311606a4db3164792c5b03339d4d108476ba361877d9f9246fc9b34d9"} Jul 17 01:13:58.163818 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.163784 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerStarted","Data":"ad51c4eff1e3627203bc07c1db05a789f227a073f2b050d5f52df4092116fbdc"} Jul 17 01:13:58.167788 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.167757 2564 generic.go:356] "Generic (PLEG): container finished" podID="7e498e2dbaf2e1c392ffcb629d838109" containerID="d1862cb385003c426c7da014939d7f15e75d52dd508740c85ff8260881b3780b" exitCode=0 Jul 17 01:13:58.167938 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.167831 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" event={"ID":"7e498e2dbaf2e1c392ffcb629d838109","Type":"ContainerDied","Data":"d1862cb385003c426c7da014939d7f15e75d52dd508740c85ff8260881b3780b"} Jul 17 01:13:58.171532 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.171503 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7hnr7" event={"ID":"991a0470-8bff-4780-8a40-693299c05626","Type":"ContainerStarted","Data":"3c1cff96f21960fae7b7760f68947149ac27b0c18ea6f971a7a677109134e74d"} Jul 17 01:13:58.175312 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.175286 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kjqjm" event={"ID":"d65398b0-bacb-4d38-b91b-87d7b21beb1f","Type":"ContainerStarted","Data":"80d71893ea0d04debefa9598d37c911f57c143f0b6b69a20df77a90f8e5f253a"} Jul 17 01:13:58.182204 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.179381 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal" event={"ID":"584482ea035b563238d63a3c04b310b9","Type":"ContainerStarted","Data":"a69ea049ba92ce8d9aa6b4a1f7ecb4040eef8dcec67fc4bc3f3bb8f2856419e4"} Jul 17 01:13:58.186275 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.186238 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerStarted","Data":"859d58ca2406d6392eacf737601f3469431af22e25064ba0bdf3d3e442de7919"} Jul 17 01:13:58.189086 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.189014 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8pf7" event={"ID":"5622d70d-5b2e-43f6-9982-b8a931543303","Type":"ContainerStarted","Data":"29db67ab684a6cb28cc60d4366a9ba77eec3bf91097079c8a9ff303d1428a8e3"} Jul 17 01:13:58.193735 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.193658 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-285tz" event={"ID":"b5a2b823-b37e-4a93-8e0d-00b83715ec5f","Type":"ContainerStarted","Data":"6ab2483324f78b02916f00b3694a0dfd7302e77b95720e94cef6868d3e88f9a1"} Jul 17 01:13:58.202103 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:58.202068 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l9qrz" event={"ID":"5602e00b-be85-4eca-adbc-e7852aed270a","Type":"ContainerStarted","Data":"2702c5ef67b80b40e1ff7b5248a7771184f94e475d207c614cce74bfaf3a46cb"} Jul 17 01:13:59.128955 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:59.128271 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:13:59.128955 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.128414 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:13:59.128955 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:59.128800 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:59.128955 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.128909 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:13:59.220577 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:59.220530 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" event={"ID":"7e498e2dbaf2e1c392ffcb629d838109","Type":"ContainerStarted","Data":"1bc5bd90a70ed4e8fdd2388bc055b13facc64fc8ea046a33827bda560f6a78f2"} Jul 17 01:13:59.238956 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:59.238896 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-58.ec2.internal" podStartSLOduration=4.238875494 podStartE2EDuration="4.238875494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:58.197974134 +0000 UTC m=+4.575898543" watchObservedRunningTime="2026-07-17 01:13:59.238875494 +0000 UTC m=+5.616799906" Jul 17 01:13:59.728340 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:59.726782 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:13:59.728340 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.726958 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 01:13:59.728340 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.727113 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret podName:a5f557eb-8315-4fb5-803c-9d0c7961cb86 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:03.72708492 +0000 UTC m=+10.105009323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret") pod "global-pull-secret-syncer-zgf6x" (UID: "a5f557eb-8315-4fb5-803c-9d0c7961cb86") : object "kube-system"/"original-pull-secret" not registered Jul 17 01:13:59.828532 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:59.827685 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:13:59.828532 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:13:59.827747 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:13:59.828532 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.827921 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 01:13:59.828532 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.827943 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 01:13:59.828532 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.827957 2564 projected.go:196] Error preparing data for projected volume kube-api-access-j9nhm for pod openshift-network-diagnostics/network-check-target-64dhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:13:59.828532 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.828016 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm podName:d7ed04a1-66ad-456b-93b4-1ddf01000212 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:03.827997553 +0000 UTC m=+10.205921946 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9nhm" (UniqueName: "kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm") pod "network-check-target-64dhp" (UID: "d7ed04a1-66ad-456b-93b4-1ddf01000212") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:13:59.828532 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.828423 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:13:59.828532 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:13:59.828472 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs podName:078ede28-d315-4b2a-862c-daeed6abdb41 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:03.828456518 +0000 UTC m=+10.206380931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs") pod "network-metrics-daemon-qjf6m" (UID: "078ede28-d315-4b2a-862c-daeed6abdb41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:14:00.128826 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:00.128264 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:00.128826 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:00.128410 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:01.128520 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:01.127999 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:01.128520 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:01.128020 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:01.128520 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:01.128132 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:01.128520 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:01.128245 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:02.127789 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:02.127757 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:02.127971 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:02.127888 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:03.128396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:03.127759 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:03.128396 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.127900 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:03.128396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:03.128269 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:03.128396 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.128354 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:03.765277 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:03.765242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:03.765468 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.765400 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 01:14:03.765468 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.765465 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret podName:a5f557eb-8315-4fb5-803c-9d0c7961cb86 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:11.765449804 +0000 UTC m=+18.143374189 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret") pod "global-pull-secret-syncer-zgf6x" (UID: "a5f557eb-8315-4fb5-803c-9d0c7961cb86") : object "kube-system"/"original-pull-secret" not registered Jul 17 01:14:03.865604 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:03.865561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:03.865783 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:03.865617 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:03.865783 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.865710 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:14:03.865783 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.865734 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 01:14:03.865783 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.865747 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 01:14:03.865783 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.865760 2564 projected.go:196] Error preparing data for projected volume kube-api-access-j9nhm for pod openshift-network-diagnostics/network-check-target-64dhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:14:03.865783 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.865782 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs podName:078ede28-d315-4b2a-862c-daeed6abdb41 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:11.865762583 +0000 UTC m=+18.243686983 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs") pod "network-metrics-daemon-qjf6m" (UID: "078ede28-d315-4b2a-862c-daeed6abdb41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:14:03.866073 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:03.865802 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm podName:d7ed04a1-66ad-456b-93b4-1ddf01000212 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:11.865789346 +0000 UTC m=+18.243713731 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9nhm" (UniqueName: "kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm") pod "network-check-target-64dhp" (UID: "d7ed04a1-66ad-456b-93b4-1ddf01000212") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:14:04.129016 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:04.128933 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:04.129494 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:04.129044 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:05.127328 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:05.127294 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:05.127493 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:05.127427 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:05.127848 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:05.127819 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:05.127986 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:05.127924 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:06.129951 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:06.129917 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:06.130409 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:06.130037 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:07.127871 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:07.127828 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:07.128052 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:07.127832 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:07.128052 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:07.127976 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:07.128052 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:07.128042 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:08.128132 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:08.128098 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:08.128613 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:08.128235 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:09.127869 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:09.127816 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:09.127869 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:09.127872 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:09.128266 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:09.128006 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:09.128266 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:09.128105 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:10.129598 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:10.129567 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:10.130024 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:10.129704 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:11.127348 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:11.127314 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:11.127515 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:11.127362 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:11.127515 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.127444 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:11.127624 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.127577 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:11.827548 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:11.827148 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:11.827548 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.827302 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 01:14:11.827548 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.827493 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret podName:a5f557eb-8315-4fb5-803c-9d0c7961cb86 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:27.827472505 +0000 UTC m=+34.205396912 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret") pod "global-pull-secret-syncer-zgf6x" (UID: "a5f557eb-8315-4fb5-803c-9d0c7961cb86") : object "kube-system"/"original-pull-secret" not registered Jul 17 01:14:11.928960 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:11.928924 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:11.929129 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:11.928992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:11.929212 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.929184 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 01:14:11.929275 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.929213 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 01:14:11.929275 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.929238 2564 projected.go:196] Error preparing data for projected volume kube-api-access-j9nhm for pod openshift-network-diagnostics/network-check-target-64dhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:14:11.929275 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.929260 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:14:11.929401 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.929309 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm podName:d7ed04a1-66ad-456b-93b4-1ddf01000212 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:27.929281901 +0000 UTC m=+34.307206301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9nhm" (UniqueName: "kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm") pod "network-check-target-64dhp" (UID: "d7ed04a1-66ad-456b-93b4-1ddf01000212") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:14:11.929401 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:11.929335 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs podName:078ede28-d315-4b2a-862c-daeed6abdb41 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:27.929317379 +0000 UTC m=+34.307241780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs") pod "network-metrics-daemon-qjf6m" (UID: "078ede28-d315-4b2a-862c-daeed6abdb41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:14:12.128406 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.128375 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:12.128576 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:12.128514 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:12.252387 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.252348 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l9qrz" event={"ID":"5602e00b-be85-4eca-adbc-e7852aed270a","Type":"ContainerStarted","Data":"073d9fc93e10ca2bf5c16b2c31798e502595abc11a69411de184c8efbe6237ab"} Jul 17 01:14:12.253940 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.253907 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jpvtd" event={"ID":"86d44f9e-b02e-40ec-863f-b528b36e5f68","Type":"ContainerStarted","Data":"ab7e71a57095335eae9cff892e1545f0e09457b7f86290ea4ba14e3b5029dc7f"} Jul 17 01:14:12.255580 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.255544 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" event={"ID":"85108d9c-6817-4c3c-bf3f-6fee0ebbe847","Type":"ContainerStarted","Data":"3c9e31f18a122492983f62246ec18ab73b4c2b97b40c326b657281d0a7e866b4"} Jul 17 01:14:12.257356 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.257018 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7hnr7" event={"ID":"991a0470-8bff-4780-8a40-693299c05626","Type":"ContainerStarted","Data":"a07465e067813c95a653b2c3e19e7de900ffb811a60a4628ff2cb6b15a649e8a"} Jul 17 01:14:12.258754 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.258663 2564 generic.go:356] "Generic (PLEG): container finished" podID="d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb" containerID="e2a708a652a3cd5f648658e1e746a7ba211bf89fb93acda5ffbf18746fc0613c" exitCode=0 Jul 17 01:14:12.258754 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.258708 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerDied","Data":"e2a708a652a3cd5f648658e1e746a7ba211bf89fb93acda5ffbf18746fc0613c"} Jul 17 01:14:12.260507 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.260483 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z8pf7" event={"ID":"5622d70d-5b2e-43f6-9982-b8a931543303","Type":"ContainerStarted","Data":"a7456d97e5cc9ae2ce3ddebcb1d4ddb3d97268a58e17c38957158748da6d0289"} Jul 17 01:14:12.262353 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.262327 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-285tz" event={"ID":"b5a2b823-b37e-4a93-8e0d-00b83715ec5f","Type":"ContainerStarted","Data":"82a3af1f2d5623ab14c3e9c1ed6de724e626efd2ae1e62a7eb9dbab567fc548b"} Jul 17 01:14:12.265917 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.265876 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-58.ec2.internal" podStartSLOduration=17.265863285000002 podStartE2EDuration="17.265863285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:13:59.238478063 +0000 UTC m=+5.616402472" watchObservedRunningTime="2026-07-17 01:14:12.265863285 +0000 UTC m=+18.643787694" Jul 17 01:14:12.278611 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.278548 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l9qrz" podStartSLOduration=4.072030828 podStartE2EDuration="18.27852672s" totalImagesPullingTime="14.206495892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:12.265933599 +0000 UTC m=+18.643858012" watchObservedRunningTime="2026-07-17 01:14:12.27852672 +0000 UTC m=+18.656451119" Jul 17 01:14:12.279436 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.279393 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jpvtd" podStartSLOduration=4.069170064 podStartE2EDuration="18.27937781s" totalImagesPullingTime="14.210207746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:12.278616657 +0000 UTC m=+18.656541066" watchObservedRunningTime="2026-07-17 01:14:12.27937781 +0000 UTC m=+18.657302219" Jul 17 01:14:12.294427 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.294337 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z8pf7" podStartSLOduration=3.873447717 podStartE2EDuration="18.294321072s" totalImagesPullingTime="14.420873355s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:12.293373363 +0000 UTC m=+18.671297765" watchObservedRunningTime="2026-07-17 01:14:12.294321072 +0000 UTC m=+18.672245491" Jul 17 01:14:12.325688 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.325624 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-285tz" podStartSLOduration=4.113652637 podStartE2EDuration="18.325600604s" totalImagesPullingTime="14.211947967s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:12.325480204 +0000 UTC m=+18.703404614" watchObservedRunningTime="2026-07-17 01:14:12.325600604 +0000 UTC m=+18.703525015" Jul 17 01:14:12.338827 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:12.338777 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7hnr7" podStartSLOduration=4.126124713 podStartE2EDuration="18.338757172s" totalImagesPullingTime="14.212632459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:12.338253454 +0000 UTC m=+18.716177863" watchObservedRunningTime="2026-07-17 01:14:12.338757172 +0000 UTC m=+18.716681621" Jul 17 01:14:13.128233 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:13.128195 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:13.128233 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:13.128217 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:13.129024 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:13.128309 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:13.129024 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:13.128428 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:13.266757 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:13.266665 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kjqjm" event={"ID":"d65398b0-bacb-4d38-b91b-87d7b21beb1f","Type":"ContainerStarted","Data":"20ab2751c5557d24dac129a21f43818dca3aa201d6de0b3af91c7e8145947a62"} Jul 17 01:14:13.281713 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:13.281658 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kjqjm" podStartSLOduration=5.082099376 podStartE2EDuration="19.281643383s" totalImagesPullingTime="14.199544007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:13.281113602 +0000 UTC m=+19.659038015" watchObservedRunningTime="2026-07-17 01:14:13.281643383 +0000 UTC m=+19.659567790" Jul 17 01:14:14.128231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:14.128199 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:14.128682 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:14.128311 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:15.127623 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:15.127593 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:15.127804 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:15.127593 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:15.127804 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:15.127709 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:15.127804 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:15.127773 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:15.788094 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:15.788053 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:14:15.789054 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:15.789035 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:14:16.127634 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:16.127538 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:16.127837 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:16.127696 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:16.272723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:16.272688 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:14:16.273148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:16.273126 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l9qrz" Jul 17 01:14:17.128344 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.127951 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:17.128783 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.128027 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:17.128783 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:17.128447 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:17.128783 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:17.128536 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:17.281071 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.280998 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:14:17.281416 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.281383 2564 generic.go:356] "Generic (PLEG): container finished" podID="44020bfd-3d70-4b54-8782-12b022f858bb" containerID="4ca5dc2c205bf2e5c9a1b852d32061365f1d4c9519e2020b9a7f5a141ea5573d" exitCode=1 Jul 17 01:14:17.281947 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.281922 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerStarted","Data":"e14bdd801a77f221875b10a014aaf01dd0d411fc38667701eb023e15ca896ea5"} Jul 17 01:14:17.282036 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.282006 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerStarted","Data":"92517ff905bc1438f21533620d26861315a873912ef044aaf4120ba273715695"} Jul 17 01:14:17.282097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.282086 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerStarted","Data":"fb5465715a123ce0d3a5553ed86686e195fe42351efdd161571b1d7c3ddc4a57"} Jul 17 01:14:17.282148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.282133 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerStarted","Data":"a28bbbfc2a0e678288869de6ecda78fc618ccd8977f3d0e31811a33bef5ac1bd"} Jul 17 01:14:17.282220 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.282198 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerDied","Data":"4ca5dc2c205bf2e5c9a1b852d32061365f1d4c9519e2020b9a7f5a141ea5573d"} Jul 17 01:14:17.282273 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.282249 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerStarted","Data":"327233c57d4c139e204b7b5f49059405d127c43a8d8afc14db337979f41cfb49"} Jul 17 01:14:17.348895 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:17.348872 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 17 01:14:18.043370 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:18.043236 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T01:14:17.348890237Z","UUID":"63dee4f8-5858-4e79-b48b-32f4b5bde07e","Handler":null,"Name":"","Endpoint":""} Jul 17 01:14:18.045018 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:18.044995 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 17 01:14:18.045132 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:18.045040 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 17 01:14:18.128242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:18.128208 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:18.128396 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:18.128327 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:18.286427 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:18.286382 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" event={"ID":"85108d9c-6817-4c3c-bf3f-6fee0ebbe847","Type":"ContainerStarted","Data":"89acee402371acfee97ca3b05fa654d40371891bd2e0e9250b80b30de75b1eda"} Jul 17 01:14:19.128022 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:19.127944 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:19.128022 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:19.127982 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:19.128253 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:19.128068 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:19.128253 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:19.128233 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:19.291647 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:19.291598 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" event={"ID":"85108d9c-6817-4c3c-bf3f-6fee0ebbe847","Type":"ContainerStarted","Data":"9fd1d82c43c8adeeac396a87310e59394dd318de37efc1a218c088c4c3af6f12"} Jul 17 01:14:19.309505 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:19.309455 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9tjqk" podStartSLOduration=4.224162033 podStartE2EDuration="25.309434719s" totalImagesPullingTime="21.085272686s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:19.30928437 +0000 UTC m=+25.687208779" watchObservedRunningTime="2026-07-17 01:14:19.309434719 +0000 UTC m=+25.687359124" Jul 17 01:14:20.127384 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:20.127353 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:20.127544 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:20.127477 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:21.127949 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:21.127875 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:21.127949 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:21.127921 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:21.128428 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:21.128008 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:21.128428 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:21.128122 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:21.299255 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:21.298521 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerStarted","Data":"d93a8c48cbc31c255cb95d74fc5f9e4229d3a6bb5b60788dc828c2cc62c3892a"} Jul 17 01:14:21.303210 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:21.303192 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:14:21.303529 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:21.303508 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerStarted","Data":"d591063dc73845938ba463389cb49e8f6986b21fa6d87e20245326178b572380"} Jul 17 01:14:22.127545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:22.127513 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:22.127712 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:22.127653 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:22.307942 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:22.307844 2564 generic.go:356] "Generic (PLEG): container finished" podID="d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb" containerID="d93a8c48cbc31c255cb95d74fc5f9e4229d3a6bb5b60788dc828c2cc62c3892a" exitCode=0 Jul 17 01:14:22.307942 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:22.307910 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerDied","Data":"d93a8c48cbc31c255cb95d74fc5f9e4229d3a6bb5b60788dc828c2cc62c3892a"} Jul 17 01:14:23.128046 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:23.127821 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:23.128150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:23.127833 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:23.128223 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:23.128181 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:23.128284 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:23.128261 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:23.313611 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:23.313577 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:14:23.314089 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:23.313940 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerStarted","Data":"ef155a2da8aea10df151367c6f974e915b09e3dfaaf3d24bef1f076d2c95d92e"} Jul 17 01:14:23.314231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:23.314215 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:14:23.314433 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:23.314407 2564 scope.go:117] "RemoveContainer" containerID="4ca5dc2c205bf2e5c9a1b852d32061365f1d4c9519e2020b9a7f5a141ea5573d" Jul 17 01:14:23.316235 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:23.316156 2564 generic.go:356] "Generic (PLEG): container finished" podID="d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb" containerID="04e8077623b6faf6816427acd73380bba8299dbc636d1ec8f74e4848dbd75c12" exitCode=0 Jul 17 01:14:23.316235 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:23.316198 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerDied","Data":"04e8077623b6faf6816427acd73380bba8299dbc636d1ec8f74e4848dbd75c12"} Jul 17 01:14:23.331411 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:23.331391 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:14:24.129702 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:24.129662 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:24.129893 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:24.129811 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:24.322910 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:24.322881 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:14:24.323384 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:24.323355 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" event={"ID":"44020bfd-3d70-4b54-8782-12b022f858bb","Type":"ContainerStarted","Data":"c8c3d149adc5b0f74469487d1e1a8057558a25c5c8ff98a287dc0936c7e511a8"} Jul 17 01:14:24.323793 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:24.323772 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:14:24.323951 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:24.323925 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:14:24.345090 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:24.345062 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:14:24.369594 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:24.369541 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" podStartSLOduration=11.29463193 podStartE2EDuration="30.36952147s" totalImagesPullingTime="19.07488954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:24.366047989 +0000 UTC m=+30.743972397" watchObservedRunningTime="2026-07-17 01:14:24.36952147 +0000 UTC m=+30.747445879" Jul 17 01:14:25.128289 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.128103 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:25.128405 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.128189 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:25.128405 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:25.128382 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:25.128466 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:25.128441 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:25.274117 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.274088 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zgf6x"] Jul 17 01:14:25.274270 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.274214 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:25.274337 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:25.274317 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:25.276987 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.276960 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-64dhp"] Jul 17 01:14:25.285808 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.285737 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qjf6m"] Jul 17 01:14:25.328851 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.328821 2564 generic.go:356] "Generic (PLEG): container finished" podID="d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb" containerID="26ccccc0cce51e6822b69bafa0ef87aa725638070076d656205036e21ded40a6" exitCode=0 Jul 17 01:14:25.329300 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.328965 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:25.329300 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.328977 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerDied","Data":"26ccccc0cce51e6822b69bafa0ef87aa725638070076d656205036e21ded40a6"} Jul 17 01:14:25.329300 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:25.329057 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:25.329300 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:25.329064 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:25.329529 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:25.329499 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:27.128064 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:27.128026 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:27.128064 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:27.128063 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:27.128672 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:27.128073 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:27.128672 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.128155 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zgf6x" podUID="a5f557eb-8315-4fb5-803c-9d0c7961cb86" Jul 17 01:14:27.128672 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.128268 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64dhp" podUID="d7ed04a1-66ad-456b-93b4-1ddf01000212" Jul 17 01:14:27.128672 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.128358 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-qjf6m" podUID="078ede28-d315-4b2a-862c-daeed6abdb41" Jul 17 01:14:27.845061 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:27.845012 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:27.845460 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.845150 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 01:14:27.845460 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.845235 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret podName:a5f557eb-8315-4fb5-803c-9d0c7961cb86 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:59.845217813 +0000 UTC m=+66.223142200 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret") pod "global-pull-secret-syncer-zgf6x" (UID: "a5f557eb-8315-4fb5-803c-9d0c7961cb86") : object "kube-system"/"original-pull-secret" not registered Jul 17 01:14:27.946074 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:27.946032 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:27.946271 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:27.946093 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:27.946271 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.946244 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:14:27.946388 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.946276 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 01:14:27.946388 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.946293 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 01:14:27.946388 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.946306 2564 projected.go:196] Error preparing data for projected volume kube-api-access-j9nhm for pod openshift-network-diagnostics/network-check-target-64dhp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:14:27.946388 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.946325 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs podName:078ede28-d315-4b2a-862c-daeed6abdb41 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:59.946300768 +0000 UTC m=+66.324225172 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs") pod "network-metrics-daemon-qjf6m" (UID: "078ede28-d315-4b2a-862c-daeed6abdb41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 01:14:27.946388 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:14:27.946352 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm podName:d7ed04a1-66ad-456b-93b4-1ddf01000212 nodeName:}" failed. No retries permitted until 2026-07-17 01:14:59.946337205 +0000 UTC m=+66.324261611 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-j9nhm" (UniqueName: "kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm") pod "network-check-target-64dhp" (UID: "d7ed04a1-66ad-456b-93b4-1ddf01000212") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 01:14:28.461031 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.459593 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-58.ec2.internal" event="NodeReady" Jul 17 01:14:28.461031 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.459735 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 17 01:14:28.496025 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.495991 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-fd4c74d88-zx8f2"] Jul 17 01:14:28.516352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.516314 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q7n29"] Jul 17 01:14:28.516519 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.516452 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.524873 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.524841 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 17 01:14:28.525003 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.524851 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 17 01:14:28.527196 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.526663 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 17 01:14:28.527196 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.526735 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sjlm6\"" Jul 17 01:14:28.530225 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.530201 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nx2gc"] Jul 17 01:14:28.530334 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.530318 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.540260 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.540241 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lh9xl\"" Jul 17 01:14:28.540413 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.540387 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 17 01:14:28.540520 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.540357 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 17 01:14:28.542933 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.542912 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fd4c74d88-zx8f2"] Jul 17 01:14:28.543041 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.542941 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q7n29"] Jul 17 01:14:28.543041 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.542959 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nx2gc"] Jul 17 01:14:28.543111 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.543046 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.545585 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.545562 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 17 01:14:28.545699 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.545687 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h47bb\"" Jul 17 01:14:28.545795 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.545776 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 17 01:14:28.545963 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.545950 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 17 01:14:28.550147 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.550126 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 17 01:14:28.551404 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.551386 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 17 01:14:28.609236 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.609206 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2b788"] Jul 17 01:14:28.631507 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.631478 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2b788"] Jul 17 01:14:28.631679 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.631583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2b788" Jul 17 01:14:28.634512 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.634488 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-php8q\"" Jul 17 01:14:28.634795 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.634771 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 17 01:14:28.634927 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.634876 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 17 01:14:28.635148 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.635131 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 17 01:14:28.651402 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27278acb-c108-4aab-ae3d-24c82e936f6e-metrics-tls\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.651545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651414 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-registry-tls\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.651545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651442 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-ca-trust-extracted\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.651545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-trusted-ca\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.651545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-image-registry-private-configuration\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.651545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651500 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-registry-certificates\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.651545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651517 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27278acb-c108-4aab-ae3d-24c82e936f6e-tmp-dir\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.651545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651533 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27bba4b4-b060-45b3-9136-39d03b5e2c00-data-volume\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.651832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651552 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27bba4b4-b060-45b3-9136-39d03b5e2c00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.651832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651596 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27278acb-c108-4aab-ae3d-24c82e936f6e-config-volume\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.651832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27bba4b4-b060-45b3-9136-39d03b5e2c00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.651832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651640 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-installation-pull-secrets\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.651832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651734 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v22s6\" (UniqueName: \"kubernetes.io/projected/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-kube-api-access-v22s6\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.651832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651754 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqjxm\" (UniqueName: \"kubernetes.io/projected/27278acb-c108-4aab-ae3d-24c82e936f6e-kube-api-access-cqjxm\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.651832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651773 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27bba4b4-b060-45b3-9136-39d03b5e2c00-crio-socket\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.651832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651789 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-bound-sa-token\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.652131 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.651842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8bfc\" (UniqueName: \"kubernetes.io/projected/27bba4b4-b060-45b3-9136-39d03b5e2c00-kube-api-access-q8bfc\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.752876 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.752787 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v22s6\" (UniqueName: \"kubernetes.io/projected/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-kube-api-access-v22s6\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.752876 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.752830 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fccaee8-8ef0-4180-90ae-4a6540066918-cert\") pod \"ingress-canary-2b788\" (UID: \"8fccaee8-8ef0-4180-90ae-4a6540066918\") " pod="openshift-ingress-canary/ingress-canary-2b788" Jul 17 01:14:28.752876 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.752862 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqjxm\" (UniqueName: \"kubernetes.io/projected/27278acb-c108-4aab-ae3d-24c82e936f6e-kube-api-access-cqjxm\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.753120 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.752890 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27bba4b4-b060-45b3-9136-39d03b5e2c00-crio-socket\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.753120 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.752925 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-bound-sa-token\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.753120 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfm4v\" (UniqueName: \"kubernetes.io/projected/8fccaee8-8ef0-4180-90ae-4a6540066918-kube-api-access-kfm4v\") pod \"ingress-canary-2b788\" (UID: \"8fccaee8-8ef0-4180-90ae-4a6540066918\") " pod="openshift-ingress-canary/ingress-canary-2b788" Jul 17 01:14:28.753276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8bfc\" (UniqueName: \"kubernetes.io/projected/27bba4b4-b060-45b3-9136-39d03b5e2c00-kube-api-access-q8bfc\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.753276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753236 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27278acb-c108-4aab-ae3d-24c82e936f6e-metrics-tls\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.753276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-registry-tls\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.753412 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753295 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-ca-trust-extracted\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.753412 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753318 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-trusted-ca\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.753412 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753335 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/27bba4b4-b060-45b3-9136-39d03b5e2c00-crio-socket\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.753412 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753346 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-image-registry-private-configuration\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.753412 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753379 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-registry-certificates\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.753412 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753404 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27278acb-c108-4aab-ae3d-24c82e936f6e-tmp-dir\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.753673 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27bba4b4-b060-45b3-9136-39d03b5e2c00-data-volume\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.753673 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753452 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27bba4b4-b060-45b3-9136-39d03b5e2c00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.753673 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753477 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27278acb-c108-4aab-ae3d-24c82e936f6e-config-volume\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.753673 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27bba4b4-b060-45b3-9136-39d03b5e2c00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.753673 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753530 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-installation-pull-secrets\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.753888 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.753862 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-ca-trust-extracted\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.754202 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.754151 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/27bba4b4-b060-45b3-9136-39d03b5e2c00-data-volume\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.754656 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.754447 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-trusted-ca\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.754656 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.754479 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/27278acb-c108-4aab-ae3d-24c82e936f6e-config-volume\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.754889 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.754868 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-registry-certificates\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.754945 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.754901 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/27bba4b4-b060-45b3-9136-39d03b5e2c00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.755079 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.755052 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27278acb-c108-4aab-ae3d-24c82e936f6e-tmp-dir\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.758203 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.758179 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/27bba4b4-b060-45b3-9136-39d03b5e2c00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.758534 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.758514 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/27278acb-c108-4aab-ae3d-24c82e936f6e-metrics-tls\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.758600 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.758587 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-installation-pull-secrets\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.758953 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.758928 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-registry-tls\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.759067 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.758958 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-image-registry-private-configuration\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.763940 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.763912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8bfc\" (UniqueName: \"kubernetes.io/projected/27bba4b4-b060-45b3-9136-39d03b5e2c00-kube-api-access-q8bfc\") pod \"insights-runtime-extractor-nx2gc\" (UID: \"27bba4b4-b060-45b3-9136-39d03b5e2c00\") " pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.764060 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.764002 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-bound-sa-token\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.764848 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.764822 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v22s6\" (UniqueName: \"kubernetes.io/projected/40b01a30-59e8-4950-a1bc-3ce63dc19e9d-kube-api-access-v22s6\") pod \"image-registry-fd4c74d88-zx8f2\" (UID: \"40b01a30-59e8-4950-a1bc-3ce63dc19e9d\") " pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.764940 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.764918 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqjxm\" (UniqueName: \"kubernetes.io/projected/27278acb-c108-4aab-ae3d-24c82e936f6e-kube-api-access-cqjxm\") pod \"dns-default-q7n29\" (UID: \"27278acb-c108-4aab-ae3d-24c82e936f6e\") " pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.830004 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.829954 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:28.843128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.843091 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:28.852884 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.852854 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nx2gc" Jul 17 01:14:28.854135 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.854109 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fccaee8-8ef0-4180-90ae-4a6540066918-cert\") pod \"ingress-canary-2b788\" (UID: \"8fccaee8-8ef0-4180-90ae-4a6540066918\") " pod="openshift-ingress-canary/ingress-canary-2b788" Jul 17 01:14:28.854282 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.854184 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfm4v\" (UniqueName: \"kubernetes.io/projected/8fccaee8-8ef0-4180-90ae-4a6540066918-kube-api-access-kfm4v\") pod \"ingress-canary-2b788\" (UID: \"8fccaee8-8ef0-4180-90ae-4a6540066918\") " pod="openshift-ingress-canary/ingress-canary-2b788" Jul 17 01:14:28.859673 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.859650 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8fccaee8-8ef0-4180-90ae-4a6540066918-cert\") pod \"ingress-canary-2b788\" (UID: \"8fccaee8-8ef0-4180-90ae-4a6540066918\") " pod="openshift-ingress-canary/ingress-canary-2b788" Jul 17 01:14:28.862953 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.862927 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfm4v\" (UniqueName: \"kubernetes.io/projected/8fccaee8-8ef0-4180-90ae-4a6540066918-kube-api-access-kfm4v\") pod \"ingress-canary-2b788\" (UID: \"8fccaee8-8ef0-4180-90ae-4a6540066918\") " pod="openshift-ingress-canary/ingress-canary-2b788" Jul 17 01:14:28.942522 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:28.942483 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2b788" Jul 17 01:14:29.127415 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:29.127333 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:29.127564 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:29.127346 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:29.127564 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:29.127480 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:14:29.130372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:29.130220 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 01:14:29.130372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:29.130235 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 01:14:29.130372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:29.130230 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 01:14:29.130372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:29.130281 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hzllc\"" Jul 17 01:14:29.130372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:29.130316 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 01:14:29.131285 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:29.131268 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dtqbv\"" Jul 17 01:14:30.140809 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.140617 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nx2gc"] Jul 17 01:14:30.146961 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.146935 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2b788"] Jul 17 01:14:30.150433 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.150410 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fd4c74d88-zx8f2"] Jul 17 01:14:30.153363 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.153333 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q7n29"] Jul 17 01:14:30.155801 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:30.155685 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27bba4b4_b060_45b3_9136_39d03b5e2c00.slice/crio-2d0ffe8c160739aecb96fc6cd6f7ffc01b553c9c6cee38c2bb5c604a674c9f35 WatchSource:0}: Error finding container 2d0ffe8c160739aecb96fc6cd6f7ffc01b553c9c6cee38c2bb5c604a674c9f35: Status 404 returned error can't find the container with id 2d0ffe8c160739aecb96fc6cd6f7ffc01b553c9c6cee38c2bb5c604a674c9f35 Jul 17 01:14:30.155920 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:30.155890 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fccaee8_8ef0_4180_90ae_4a6540066918.slice/crio-4f48ee81ef174c6d4f4997a316b3556a341a957d8a973528c78720ba915dfcdb WatchSource:0}: Error finding container 4f48ee81ef174c6d4f4997a316b3556a341a957d8a973528c78720ba915dfcdb: Status 404 returned error can't find the container with id 4f48ee81ef174c6d4f4997a316b3556a341a957d8a973528c78720ba915dfcdb Jul 17 01:14:30.157620 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:30.157526 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b01a30_59e8_4950_a1bc_3ce63dc19e9d.slice/crio-aca95e70a2a628df872118255ed8c899b808742dc2895a1a822753af4cb36f9b WatchSource:0}: Error finding container aca95e70a2a628df872118255ed8c899b808742dc2895a1a822753af4cb36f9b: Status 404 returned error can't find the container with id aca95e70a2a628df872118255ed8c899b808742dc2895a1a822753af4cb36f9b Jul 17 01:14:30.158117 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:30.157830 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27278acb_c108_4aab_ae3d_24c82e936f6e.slice/crio-111eb5f3093c07543467cb9e97b6aa3414e5878483caade45ab56961180c4b1d WatchSource:0}: Error finding container 111eb5f3093c07543467cb9e97b6aa3414e5878483caade45ab56961180c4b1d: Status 404 returned error can't find the container with id 111eb5f3093c07543467cb9e97b6aa3414e5878483caade45ab56961180c4b1d Jul 17 01:14:30.345389 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.345349 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2b788" event={"ID":"8fccaee8-8ef0-4180-90ae-4a6540066918","Type":"ContainerStarted","Data":"4f48ee81ef174c6d4f4997a316b3556a341a957d8a973528c78720ba915dfcdb"} Jul 17 01:14:30.346519 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.346485 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nx2gc" event={"ID":"27bba4b4-b060-45b3-9136-39d03b5e2c00","Type":"ContainerStarted","Data":"2d0ffe8c160739aecb96fc6cd6f7ffc01b553c9c6cee38c2bb5c604a674c9f35"} Jul 17 01:14:30.347604 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.347573 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q7n29" event={"ID":"27278acb-c108-4aab-ae3d-24c82e936f6e","Type":"ContainerStarted","Data":"111eb5f3093c07543467cb9e97b6aa3414e5878483caade45ab56961180c4b1d"} Jul 17 01:14:30.350210 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.350191 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerStarted","Data":"bfcb9c93388811f06cb2d33d75c70110122ba9df786fd14df6cda4c515b7b85f"} Jul 17 01:14:30.351610 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.351589 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" event={"ID":"40b01a30-59e8-4950-a1bc-3ce63dc19e9d","Type":"ContainerStarted","Data":"6804d005625bc66f102cc3122d8e08eea6ac1e743fdbec6d3e7299198529a709"} Jul 17 01:14:30.351698 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.351625 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" event={"ID":"40b01a30-59e8-4950-a1bc-3ce63dc19e9d","Type":"ContainerStarted","Data":"aca95e70a2a628df872118255ed8c899b808742dc2895a1a822753af4cb36f9b"} Jul 17 01:14:30.351805 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.351789 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:30.387122 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:30.387076 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" podStartSLOduration=6.387059921 podStartE2EDuration="6.387059921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:30.386440575 +0000 UTC m=+36.764364986" watchObservedRunningTime="2026-07-17 01:14:30.387059921 +0000 UTC m=+36.764984307" Jul 17 01:14:31.359137 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:31.359095 2564 generic.go:356] "Generic (PLEG): container finished" podID="d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb" containerID="bfcb9c93388811f06cb2d33d75c70110122ba9df786fd14df6cda4c515b7b85f" exitCode=0 Jul 17 01:14:31.360282 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:31.360259 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerDied","Data":"bfcb9c93388811f06cb2d33d75c70110122ba9df786fd14df6cda4c515b7b85f"} Jul 17 01:14:32.366458 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:32.366421 2564 generic.go:356] "Generic (PLEG): container finished" podID="d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb" containerID="130ea10641530a95a9c17b422ad3dcd73f9b3f4e236d9b61b894da33269d045a" exitCode=0 Jul 17 01:14:32.366847 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:32.366479 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerDied","Data":"130ea10641530a95a9c17b422ad3dcd73f9b3f4e236d9b61b894da33269d045a"} Jul 17 01:14:34.600900 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:34.600864 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-7q26l"] Jul 17 01:14:34.739634 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:34.739592 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-7q26l"] Jul 17 01:14:34.739795 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:34.739768 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-7q26l" Jul 17 01:14:34.742576 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:34.742556 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-hwjvr\"" Jul 17 01:14:34.742576 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:34.742565 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 17 01:14:34.742576 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:34.742574 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 17 01:14:34.908903 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:34.908869 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vrnw\" (UniqueName: \"kubernetes.io/projected/36908188-7b90-4fa5-b63d-56cf2c3edf48-kube-api-access-9vrnw\") pod \"downloads-5bc67dcd46-7q26l\" (UID: \"36908188-7b90-4fa5-b63d-56cf2c3edf48\") " pod="openshift-console/downloads-5bc67dcd46-7q26l" Jul 17 01:14:35.009458 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.009423 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vrnw\" (UniqueName: \"kubernetes.io/projected/36908188-7b90-4fa5-b63d-56cf2c3edf48-kube-api-access-9vrnw\") pod \"downloads-5bc67dcd46-7q26l\" (UID: \"36908188-7b90-4fa5-b63d-56cf2c3edf48\") " pod="openshift-console/downloads-5bc67dcd46-7q26l" Jul 17 01:14:35.019004 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.018977 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vrnw\" (UniqueName: \"kubernetes.io/projected/36908188-7b90-4fa5-b63d-56cf2c3edf48-kube-api-access-9vrnw\") pod \"downloads-5bc67dcd46-7q26l\" (UID: \"36908188-7b90-4fa5-b63d-56cf2c3edf48\") " pod="openshift-console/downloads-5bc67dcd46-7q26l" Jul 17 01:14:35.051407 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.051380 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-7q26l" Jul 17 01:14:35.299366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.298265 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt"] Jul 17 01:14:35.362497 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.362456 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt"] Jul 17 01:14:35.362656 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.362555 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.365253 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.365225 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 17 01:14:35.365386 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.365285 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 17 01:14:35.365579 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.365562 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 17 01:14:35.367343 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.366461 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 17 01:14:35.367343 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.366480 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 17 01:14:35.367343 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.366540 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-g76nx\"" Jul 17 01:14:35.514599 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.514557 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np4s8\" (UniqueName: \"kubernetes.io/projected/9f101251-b498-49bd-b934-fc4fdb6f9110-kube-api-access-np4s8\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.514599 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.514605 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f101251-b498-49bd-b934-fc4fdb6f9110-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.514853 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.514629 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f101251-b498-49bd-b934-fc4fdb6f9110-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.514853 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.514646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f101251-b498-49bd-b934-fc4fdb6f9110-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.615552 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.615467 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np4s8\" (UniqueName: \"kubernetes.io/projected/9f101251-b498-49bd-b934-fc4fdb6f9110-kube-api-access-np4s8\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.615552 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.615524 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f101251-b498-49bd-b934-fc4fdb6f9110-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.615962 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.615558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f101251-b498-49bd-b934-fc4fdb6f9110-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.615962 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.615585 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f101251-b498-49bd-b934-fc4fdb6f9110-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.616198 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.616157 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9f101251-b498-49bd-b934-fc4fdb6f9110-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.618027 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.618004 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9f101251-b498-49bd-b934-fc4fdb6f9110-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.618116 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.618010 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9f101251-b498-49bd-b934-fc4fdb6f9110-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.624016 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.623991 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np4s8\" (UniqueName: \"kubernetes.io/projected/9f101251-b498-49bd-b934-fc4fdb6f9110-kube-api-access-np4s8\") pod \"prometheus-operator-5945b5cfd-cn7dt\" (UID: \"9f101251-b498-49bd-b934-fc4fdb6f9110\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:35.673026 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:35.672988 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" Jul 17 01:14:37.084936 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:37.084912 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt"] Jul 17 01:14:37.087347 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:37.087319 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f101251_b498_49bd_b934_fc4fdb6f9110.slice/crio-436e3070a49bf9a4a9c27899cbec604548ff0c4f1fb29cf2b26d2b5e852250c9 WatchSource:0}: Error finding container 436e3070a49bf9a4a9c27899cbec604548ff0c4f1fb29cf2b26d2b5e852250c9: Status 404 returned error can't find the container with id 436e3070a49bf9a4a9c27899cbec604548ff0c4f1fb29cf2b26d2b5e852250c9 Jul 17 01:14:37.100732 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:37.100676 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-7q26l"] Jul 17 01:14:37.104695 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:37.104667 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36908188_7b90_4fa5_b63d_56cf2c3edf48.slice/crio-3f53f1ad6d2304aebdc9b35649204c93ad91adc660f52a6ed28313d14378e796 WatchSource:0}: Error finding container 3f53f1ad6d2304aebdc9b35649204c93ad91adc660f52a6ed28313d14378e796: Status 404 returned error can't find the container with id 3f53f1ad6d2304aebdc9b35649204c93ad91adc660f52a6ed28313d14378e796 Jul 17 01:14:37.383056 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:37.382942 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q7n29" event={"ID":"27278acb-c108-4aab-ae3d-24c82e936f6e","Type":"ContainerStarted","Data":"22383c7abbdee12062722b90fa307cf2a98d7641a21eab471d0c9a1b49028b60"} Jul 17 01:14:37.387494 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:37.387461 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" event={"ID":"d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb","Type":"ContainerStarted","Data":"9bed4e538545b345830af190591078201959a9a6e60de343cfe295553ac82087"} Jul 17 01:14:37.388508 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:37.388487 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-7q26l" event={"ID":"36908188-7b90-4fa5-b63d-56cf2c3edf48","Type":"ContainerStarted","Data":"3f53f1ad6d2304aebdc9b35649204c93ad91adc660f52a6ed28313d14378e796"} Jul 17 01:14:37.390048 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:37.390026 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2b788" event={"ID":"8fccaee8-8ef0-4180-90ae-4a6540066918","Type":"ContainerStarted","Data":"cf541443353b761b8446dbe3469dfc113b77596758dc70b3ae2bd316bdfec96b"} Jul 17 01:14:37.391047 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:37.391013 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" event={"ID":"9f101251-b498-49bd-b934-fc4fdb6f9110","Type":"ContainerStarted","Data":"436e3070a49bf9a4a9c27899cbec604548ff0c4f1fb29cf2b26d2b5e852250c9"} Jul 17 01:14:37.411232 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:37.411102 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-j5f6x" podStartSLOduration=13.304711019 podStartE2EDuration="43.411086517s" totalImagesPullingTime="30.106375498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:37.40976043 +0000 UTC m=+43.787684839" watchObservedRunningTime="2026-07-17 01:14:37.411086517 +0000 UTC m=+43.789010925" Jul 17 01:14:37.431665 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:37.431563 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2b788" podStartSLOduration=2.667372618 podStartE2EDuration="9.43154762s" totalImagesPullingTime="6.764175002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:37.430367822 +0000 UTC m=+43.808292232" watchObservedRunningTime="2026-07-17 01:14:37.43154762 +0000 UTC m=+43.809472008" Jul 17 01:14:38.399321 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:38.399226 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nx2gc" event={"ID":"27bba4b4-b060-45b3-9136-39d03b5e2c00","Type":"ContainerStarted","Data":"d511975f270e1871a2785083a9fd54d518ab5693ba7da5d433a973c893d9f4b6"} Jul 17 01:14:38.403544 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:38.402443 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q7n29" event={"ID":"27278acb-c108-4aab-ae3d-24c82e936f6e","Type":"ContainerStarted","Data":"52fba4ca941dce7dc1426d43dfaa132ea9c2fdc100f897cdeff388b14899890b"} Jul 17 01:14:38.420571 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:38.419640 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q7n29" podStartSLOduration=3.652814777 podStartE2EDuration="10.41961882s" totalImagesPullingTime="6.766804043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:38.41895214 +0000 UTC m=+44.796876551" watchObservedRunningTime="2026-07-17 01:14:38.41961882 +0000 UTC m=+44.797543228" Jul 17 01:14:39.406867 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:39.406840 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:40.416404 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:40.416368 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" event={"ID":"9f101251-b498-49bd-b934-fc4fdb6f9110","Type":"ContainerStarted","Data":"10f784e35be53eaf8b3c188d1cf450caee1bf714abd1c31e7b9ccc5e3686d25a"} Jul 17 01:14:42.425125 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:42.425079 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" event={"ID":"9f101251-b498-49bd-b934-fc4fdb6f9110","Type":"ContainerStarted","Data":"090845b5c2a7afdfab7cd3e1b1112aaec58e81c1afe52622e1c748050955dd5c"} Jul 17 01:14:43.430301 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:43.430259 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nx2gc" event={"ID":"27bba4b4-b060-45b3-9136-39d03b5e2c00","Type":"ContainerStarted","Data":"9d2d196ba38a9d38a813b55b1d313e5a23ec0d51feae5bab087288cad146c566"} Jul 17 01:14:44.719174 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.719117 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-cn7dt" podStartSLOduration=7.402190424 podStartE2EDuration="9.719100627s" totalImagesPullingTime="2.316910203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:42.44148618 +0000 UTC m=+48.819410589" watchObservedRunningTime="2026-07-17 01:14:44.719100627 +0000 UTC m=+51.097025043" Jul 17 01:14:44.719770 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.719464 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p9t9q"] Jul 17 01:14:44.734963 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.734929 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.737908 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.737883 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 17 01:14:44.738040 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.737948 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-k8bd4\"" Jul 17 01:14:44.738128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.738110 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 17 01:14:44.738386 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.738370 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 17 01:14:44.896043 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.896001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/31cad27c-0743-490a-bc89-144d3cdec0c4-root\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.896252 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.896055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-textfile\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.896252 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.896081 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31cad27c-0743-490a-bc89-144d3cdec0c4-metrics-client-ca\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.896252 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.896134 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-tls\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.896252 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.896192 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31cad27c-0743-490a-bc89-144d3cdec0c4-sys\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.896252 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.896241 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-wtmp\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.896514 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.896306 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.896514 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.896331 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7c5q\" (UniqueName: \"kubernetes.io/projected/31cad27c-0743-490a-bc89-144d3cdec0c4-kube-api-access-t7c5q\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.896514 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.896414 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.930866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.921939 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6696745474-9gcpg"] Jul 17 01:14:44.946964 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.946926 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6696745474-9gcpg"] Jul 17 01:14:44.947139 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.947065 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:44.949668 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.949638 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 17 01:14:44.950256 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.950054 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 17 01:14:44.950256 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.950063 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 17 01:14:44.950256 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.950087 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 17 01:14:44.950256 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.950135 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-d6l46\"" Jul 17 01:14:44.950462 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.950325 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 17 01:14:44.997929 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.997833 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-wtmp\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.997929 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.997888 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.997929 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.997914 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7c5q\" (UniqueName: \"kubernetes.io/projected/31cad27c-0743-490a-bc89-144d3cdec0c4-kube-api-access-t7c5q\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.997977 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-wtmp\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.997979 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.998029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/31cad27c-0743-490a-bc89-144d3cdec0c4-root\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.998057 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-textfile\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.998080 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31cad27c-0743-490a-bc89-144d3cdec0c4-metrics-client-ca\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.998120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-tls\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.998144 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31cad27c-0743-490a-bc89-144d3cdec0c4-sys\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.998238 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31cad27c-0743-490a-bc89-144d3cdec0c4-sys\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998690 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.998280 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/31cad27c-0743-490a-bc89-144d3cdec0c4-root\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998690 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.998622 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-textfile\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.998690 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.998621 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-accelerators-collector-config\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:44.999150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:44.999127 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31cad27c-0743-490a-bc89-144d3cdec0c4-metrics-client-ca\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:45.001366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.001340 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-tls\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:45.006804 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.006779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7c5q\" (UniqueName: \"kubernetes.io/projected/31cad27c-0743-490a-bc89-144d3cdec0c4-kube-api-access-t7c5q\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:45.007534 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.007445 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31cad27c-0743-490a-bc89-144d3cdec0c4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p9t9q\" (UID: \"31cad27c-0743-490a-bc89-144d3cdec0c4\") " pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:45.047840 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.047804 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p9t9q" Jul 17 01:14:45.090747 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:45.084187 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31cad27c_0743_490a_bc89_144d3cdec0c4.slice/crio-6f61d91846020190d003df2f8346c89a7f16765bb11ac2e4eeb72943eb18731d WatchSource:0}: Error finding container 6f61d91846020190d003df2f8346c89a7f16765bb11ac2e4eeb72943eb18731d: Status 404 returned error can't find the container with id 6f61d91846020190d003df2f8346c89a7f16765bb11ac2e4eeb72943eb18731d Jul 17 01:14:45.099558 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.099460 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm5dd\" (UniqueName: \"kubernetes.io/projected/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-kube-api-access-dm5dd\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.099558 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.099497 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-oauth-config\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.099558 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.099516 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-service-ca\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.099794 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.099610 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-oauth-serving-cert\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.099794 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.099653 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-serving-cert\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.099794 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.099703 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-config\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.200121 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.200089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm5dd\" (UniqueName: \"kubernetes.io/projected/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-kube-api-access-dm5dd\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.200121 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.200125 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-oauth-config\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.200352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.200156 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-service-ca\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.200352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.200242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-oauth-serving-cert\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.200352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.200285 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-serving-cert\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.200352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.200325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-config\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.201008 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.200983 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-config\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.202185 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.201649 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-oauth-serving-cert\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.202185 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.201695 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-service-ca\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.202921 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.202894 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-oauth-config\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.203210 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.203191 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-serving-cert\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.211195 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.211153 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm5dd\" (UniqueName: \"kubernetes.io/projected/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-kube-api-access-dm5dd\") pod \"console-6696745474-9gcpg\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.259392 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.259293 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:14:45.439376 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:45.439338 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9t9q" event={"ID":"31cad27c-0743-490a-bc89-144d3cdec0c4","Type":"ContainerStarted","Data":"6f61d91846020190d003df2f8346c89a7f16765bb11ac2e4eeb72943eb18731d"} Jul 17 01:14:47.292379 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:47.292302 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6696745474-9gcpg"] Jul 17 01:14:47.295807 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:47.295774 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab2a9ea_cab5_4aac_9dd8_1a1eea3f6354.slice/crio-bee1fefe762e00bb09951d6b80db568c6a2de0668db63cad1b9ce784b1fabc5f WatchSource:0}: Error finding container bee1fefe762e00bb09951d6b80db568c6a2de0668db63cad1b9ce784b1fabc5f: Status 404 returned error can't find the container with id bee1fefe762e00bb09951d6b80db568c6a2de0668db63cad1b9ce784b1fabc5f Jul 17 01:14:47.449407 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:47.449363 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6696745474-9gcpg" event={"ID":"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354","Type":"ContainerStarted","Data":"bee1fefe762e00bb09951d6b80db568c6a2de0668db63cad1b9ce784b1fabc5f"} Jul 17 01:14:47.451970 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:47.451936 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nx2gc" event={"ID":"27bba4b4-b060-45b3-9136-39d03b5e2c00","Type":"ContainerStarted","Data":"a8234fd0a0eb13757ac209bd9376ef6f0fa327355a50da36f81951107bbc475f"} Jul 17 01:14:47.468415 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:47.468346 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nx2gc" podStartSLOduration=2.678730036 podStartE2EDuration="19.468324904s" totalImagesPullingTime="16.789594868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:14:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:47.467415813 +0000 UTC m=+53.845340252" watchObservedRunningTime="2026-07-17 01:14:47.468324904 +0000 UTC m=+53.846249320" Jul 17 01:14:48.458069 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:48.458015 2564 generic.go:356] "Generic (PLEG): container finished" podID="31cad27c-0743-490a-bc89-144d3cdec0c4" containerID="5af3f8562b7db7a7f508d2f7e37f8cce36604f8ba1b258a87a94534e0b1a3511" exitCode=0 Jul 17 01:14:48.458501 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:48.458109 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9t9q" event={"ID":"31cad27c-0743-490a-bc89-144d3cdec0c4","Type":"ContainerDied","Data":"5af3f8562b7db7a7f508d2f7e37f8cce36604f8ba1b258a87a94534e0b1a3511"} Jul 17 01:14:49.418539 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.418508 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q7n29" Jul 17 01:14:49.465533 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.465500 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9t9q" event={"ID":"31cad27c-0743-490a-bc89-144d3cdec0c4","Type":"ContainerStarted","Data":"aaa1f08db07f04293f48ef0e870aef723ac427ac190917c793c2c69342f055fa"} Jul 17 01:14:49.465972 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.465562 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9t9q" event={"ID":"31cad27c-0743-490a-bc89-144d3cdec0c4","Type":"ContainerStarted","Data":"10c1d154639cb50474ce6d827dcca92eae0a61c260b9093744cc2fda7cbf0257"} Jul 17 01:14:49.489458 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.489395 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p9t9q" podStartSLOduration=2.775448137 podStartE2EDuration="5.489377503s" totalImagesPullingTime="2.631958743s" totalInitContainerRuntime="81.970623ms" isStatefulPod=true podCreationTimestamp="2026-07-17 01:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:49.488830776 +0000 UTC m=+55.866755198" watchObservedRunningTime="2026-07-17 01:14:49.489377503 +0000 UTC m=+55.867302002" Jul 17 01:14:49.881801 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.881719 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x"] Jul 17 01:14:49.884129 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.884107 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:49.886600 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.886565 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 17 01:14:49.886836 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.886820 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 17 01:14:49.887103 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.887088 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-b9sl4\"" Jul 17 01:14:49.887377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.887354 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 17 01:14:49.887602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.887480 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 17 01:14:49.887602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.887543 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 17 01:14:49.892604 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.892580 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 17 01:14:49.896457 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.896436 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x"] Jul 17 01:14:49.946214 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.946179 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-telemeter-client-tls\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:49.946442 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.946242 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-federate-client-tls\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:49.946442 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.946326 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:49.946442 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.946359 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjvkr\" (UniqueName: \"kubernetes.io/projected/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-kube-api-access-wjvkr\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:49.946611 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.946456 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:49.946611 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.946516 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-metrics-client-ca\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:49.946611 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.946557 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-serving-certs-ca-bundle\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:49.946754 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:49.946624 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-secret-telemeter-client\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.047128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.047092 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-metrics-client-ca\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.047128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.047131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-serving-certs-ca-bundle\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.047396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.047176 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-secret-telemeter-client\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.047396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.047256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-telemeter-client-tls\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.047396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.047292 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-federate-client-tls\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.047396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.047317 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.047396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.047343 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjvkr\" (UniqueName: \"kubernetes.io/projected/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-kube-api-access-wjvkr\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.047618 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.047399 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.048032 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.048001 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-metrics-client-ca\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.048396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.048372 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-telemeter-trusted-ca-bundle\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.049305 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.048484 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-serving-certs-ca-bundle\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.050444 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.049993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-secret-telemeter-client\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.050951 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.050569 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-federate-client-tls\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.051097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.051075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.051501 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.051478 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-telemeter-client-tls\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.056327 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.056303 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjvkr\" (UniqueName: \"kubernetes.io/projected/7484801f-51e8-4bc2-84b9-2c9d3d2aeb52-kube-api-access-wjvkr\") pod \"telemeter-client-69fb9cbc45-8qh7x\" (UID: \"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52\") " pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:50.195786 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:50.195744 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" Jul 17 01:14:51.364624 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:51.364589 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-fd4c74d88-zx8f2" Jul 17 01:14:53.709710 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.709671 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b7fff644c-6p5xn"] Jul 17 01:14:53.743204 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.743099 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b7fff644c-6p5xn"] Jul 17 01:14:53.743371 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.743252 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.752188 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.752134 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 17 01:14:53.784864 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.784827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-config\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.784864 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.784873 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-serving-cert\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.785088 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.784899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-oauth-serving-cert\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.785088 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.784986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-service-ca\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.785088 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.785050 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-trusted-ca-bundle\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.785259 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.785114 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-oauth-config\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.785259 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.785177 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grvh2\" (UniqueName: \"kubernetes.io/projected/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-kube-api-access-grvh2\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.886314 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.886277 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-trusted-ca-bundle\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.886486 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.886324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-oauth-config\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.886486 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.886356 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grvh2\" (UniqueName: \"kubernetes.io/projected/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-kube-api-access-grvh2\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.886486 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.886427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-config\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.886486 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.886457 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-serving-cert\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.886652 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.886514 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-oauth-serving-cert\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.886652 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.886563 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-service-ca\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.887330 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.887249 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-service-ca\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.887330 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.887302 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-config\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.887475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.887333 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-trusted-ca-bundle\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.887475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.887424 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-oauth-serving-cert\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.889664 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.889637 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-serving-cert\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.889775 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.889684 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-oauth-config\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:53.895144 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:53.895123 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grvh2\" (UniqueName: \"kubernetes.io/projected/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-kube-api-access-grvh2\") pod \"console-b7fff644c-6p5xn\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:54.055018 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:54.054940 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:14:56.348774 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:56.348747 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6b52l" Jul 17 01:14:58.107018 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.106981 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x"] Jul 17 01:14:58.111312 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:58.111279 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7484801f_51e8_4bc2_84b9_2c9d3d2aeb52.slice/crio-e16e1bff5717184bf5ad567362d1eba4a3c3e90e65e9db73cc905b12a45b70e5 WatchSource:0}: Error finding container e16e1bff5717184bf5ad567362d1eba4a3c3e90e65e9db73cc905b12a45b70e5: Status 404 returned error can't find the container with id e16e1bff5717184bf5ad567362d1eba4a3c3e90e65e9db73cc905b12a45b70e5 Jul 17 01:14:58.120388 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.120360 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b7fff644c-6p5xn"] Jul 17 01:14:58.123472 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:14:58.123448 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fbb12c_3ee7_4dcb_82e1_384f3ec38ba1.slice/crio-5b694102f3abb76e73f8c13243df18c7275cf1a54c7a2aa1d3d84d62ca016006 WatchSource:0}: Error finding container 5b694102f3abb76e73f8c13243df18c7275cf1a54c7a2aa1d3d84d62ca016006: Status 404 returned error can't find the container with id 5b694102f3abb76e73f8c13243df18c7275cf1a54c7a2aa1d3d84d62ca016006 Jul 17 01:14:58.502764 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.502729 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6696745474-9gcpg" event={"ID":"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354","Type":"ContainerStarted","Data":"b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae"} Jul 17 01:14:58.504558 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.504525 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" event={"ID":"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52","Type":"ContainerStarted","Data":"e16e1bff5717184bf5ad567362d1eba4a3c3e90e65e9db73cc905b12a45b70e5"} Jul 17 01:14:58.506546 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.506517 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7fff644c-6p5xn" event={"ID":"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1","Type":"ContainerStarted","Data":"7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c"} Jul 17 01:14:58.506661 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.506568 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7fff644c-6p5xn" event={"ID":"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1","Type":"ContainerStarted","Data":"5b694102f3abb76e73f8c13243df18c7275cf1a54c7a2aa1d3d84d62ca016006"} Jul 17 01:14:58.508617 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.508584 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-7q26l" event={"ID":"36908188-7b90-4fa5-b63d-56cf2c3edf48","Type":"ContainerStarted","Data":"cc70b3954782ed2665eb7930bb64c5dc8fea9fc3be5be6cd36add17fda3bc4c8"} Jul 17 01:14:58.508840 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.508815 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-7q26l" Jul 17 01:14:58.520547 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.520516 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-7q26l" Jul 17 01:14:58.523063 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.523011 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6696745474-9gcpg" podStartSLOduration=3.8694909490000002 podStartE2EDuration="14.522997469s" totalImagesPullingTime="10.65350652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:58.52189636 +0000 UTC m=+64.899820769" watchObservedRunningTime="2026-07-17 01:14:58.522997469 +0000 UTC m=+64.900921877" Jul 17 01:14:58.538779 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.538718 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-7q26l" podStartSLOduration=3.686420016 podStartE2EDuration="24.538697845s" totalImagesPullingTime="20.852277829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:58.537626209 +0000 UTC m=+64.915550618" watchObservedRunningTime="2026-07-17 01:14:58.538697845 +0000 UTC m=+64.916622254" Jul 17 01:14:58.559595 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:58.559541 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b7fff644c-6p5xn" podStartSLOduration=5.559525079 podStartE2EDuration="5.559525079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:14:58.55882059 +0000 UTC m=+64.936744998" watchObservedRunningTime="2026-07-17 01:14:58.559525079 +0000 UTC m=+64.937449545" Jul 17 01:14:59.846150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:59.845661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:59.848622 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:59.848593 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 01:14:59.859831 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:59.859798 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a5f557eb-8315-4fb5-803c-9d0c7961cb86-original-pull-secret\") pod \"global-pull-secret-syncer-zgf6x\" (UID: \"a5f557eb-8315-4fb5-803c-9d0c7961cb86\") " pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:14:59.946359 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:59.946324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:14:59.949360 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:59.949118 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 01:14:59.967895 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:14:59.967864 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/078ede28-d315-4b2a-862c-daeed6abdb41-metrics-certs\") pod \"network-metrics-daemon-qjf6m\" (UID: \"078ede28-d315-4b2a-862c-daeed6abdb41\") " pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:15:00.039967 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.039933 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zgf6x" Jul 17 01:15:00.047822 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.047631 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:15:00.048734 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.048702 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hzllc\"" Jul 17 01:15:00.050929 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.050887 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 01:15:00.057243 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.057124 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-qjf6m" Jul 17 01:15:00.061350 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.061195 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 01:15:00.072932 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.072867 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9nhm\" (UniqueName: \"kubernetes.io/projected/d7ed04a1-66ad-456b-93b4-1ddf01000212-kube-api-access-j9nhm\") pod \"network-check-target-64dhp\" (UID: \"d7ed04a1-66ad-456b-93b4-1ddf01000212\") " pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:15:00.278754 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.278721 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zgf6x"] Jul 17 01:15:00.285316 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:15:00.285278 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5f557eb_8315_4fb5_803c_9d0c7961cb86.slice/crio-d0fb451946c533cb48ac4caf5fbeed5713e32285d4c4c57b08ad8952c941baa6 WatchSource:0}: Error finding container d0fb451946c533cb48ac4caf5fbeed5713e32285d4c4c57b08ad8952c941baa6: Status 404 returned error can't find the container with id d0fb451946c533cb48ac4caf5fbeed5713e32285d4c4c57b08ad8952c941baa6 Jul 17 01:15:00.309426 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.309394 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-qjf6m"] Jul 17 01:15:00.312937 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:15:00.312905 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078ede28_d315_4b2a_862c_daeed6abdb41.slice/crio-8c2beed030a7797c152a74d918400fc824d06737538a439ccdb715dade7f927a WatchSource:0}: Error finding container 8c2beed030a7797c152a74d918400fc824d06737538a439ccdb715dade7f927a: Status 404 returned error can't find the container with id 8c2beed030a7797c152a74d918400fc824d06737538a439ccdb715dade7f927a Jul 17 01:15:00.354437 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.354408 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dtqbv\"" Jul 17 01:15:00.361866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.361796 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:15:00.519218 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.519099 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qjf6m" event={"ID":"078ede28-d315-4b2a-862c-daeed6abdb41","Type":"ContainerStarted","Data":"8c2beed030a7797c152a74d918400fc824d06737538a439ccdb715dade7f927a"} Jul 17 01:15:00.520683 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.520596 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zgf6x" event={"ID":"a5f557eb-8315-4fb5-803c-9d0c7961cb86","Type":"ContainerStarted","Data":"d0fb451946c533cb48ac4caf5fbeed5713e32285d4c4c57b08ad8952c941baa6"} Jul 17 01:15:00.544115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:00.543992 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-64dhp"] Jul 17 01:15:01.036406 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:15:01.036368 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ed04a1_66ad_456b_93b4_1ddf01000212.slice/crio-3c41caf292a7a41eb4c64a720dfe139d72447fa4140d2aef104731ca09751412 WatchSource:0}: Error finding container 3c41caf292a7a41eb4c64a720dfe139d72447fa4140d2aef104731ca09751412: Status 404 returned error can't find the container with id 3c41caf292a7a41eb4c64a720dfe139d72447fa4140d2aef104731ca09751412 Jul 17 01:15:01.529265 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:01.528331 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-64dhp" event={"ID":"d7ed04a1-66ad-456b-93b4-1ddf01000212","Type":"ContainerStarted","Data":"3c41caf292a7a41eb4c64a720dfe139d72447fa4140d2aef104731ca09751412"} Jul 17 01:15:01.534968 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:01.534933 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" event={"ID":"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52","Type":"ContainerStarted","Data":"9c9e6786744cf3e6850dad2a85780430c1d07813ca83fc0bbc128bd01a4aa497"} Jul 17 01:15:03.549340 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:03.549294 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qjf6m" event={"ID":"078ede28-d315-4b2a-862c-daeed6abdb41","Type":"ContainerStarted","Data":"46db8f9261b8dee2cfb60af000bbf729ed84504c655e0f2ddc4a7d63018087dd"} Jul 17 01:15:03.549782 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:03.549373 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-qjf6m" event={"ID":"078ede28-d315-4b2a-862c-daeed6abdb41","Type":"ContainerStarted","Data":"f32fbe4e2b37d5c2fbd0fcdb791abdf23b35730608acdc9aa8a4fad611be8bff"} Jul 17 01:15:03.566517 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:03.566313 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-qjf6m" podStartSLOduration=67.108642285 podStartE2EDuration="1m9.566290247s" totalImagesPullingTime="2.457647962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:15:03.564215244 +0000 UTC m=+69.942139659" watchObservedRunningTime="2026-07-17 01:15:03.566290247 +0000 UTC m=+69.944214656" Jul 17 01:15:04.055058 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:04.055027 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:15:04.056080 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:04.056055 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:15:04.063452 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:04.063226 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:15:04.560916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:04.560718 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:15:04.604240 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:04.603027 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6696745474-9gcpg"] Jul 17 01:15:05.259960 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:05.259882 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:15:05.564335 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:05.564217 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" event={"ID":"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52","Type":"ContainerStarted","Data":"1806569e85da8c927d54f6a0d5bacac7d7b8441c33516612647851c2ed684218"} Jul 17 01:15:05.564335 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:05.564289 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" event={"ID":"7484801f-51e8-4bc2-84b9-2c9d3d2aeb52","Type":"ContainerStarted","Data":"d8b6d1524602e92ff08fd1562fe6bc018dc0805d6b2e7f317e5b9859a0e0af9b"} Jul 17 01:15:05.590608 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:05.590535 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-69fb9cbc45-8qh7x" podStartSLOduration=10.029080709 podStartE2EDuration="16.590516508s" totalImagesPullingTime="6.561435799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:14:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:15:05.589193654 +0000 UTC m=+71.967118062" watchObservedRunningTime="2026-07-17 01:15:05.590516508 +0000 UTC m=+71.968440917" Jul 17 01:15:06.789942 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.789903 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bb8985c9d-gz5tb"] Jul 17 01:15:06.843128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.843092 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bb8985c9d-gz5tb"] Jul 17 01:15:06.843327 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.843221 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:06.919936 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.919897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-serving-cert\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:06.920128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.919953 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-config\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:06.920128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.920013 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-oauth-serving-cert\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:06.920128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.920038 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-trusted-ca-bundle\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:06.920128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.920052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ftf4\" (UniqueName: \"kubernetes.io/projected/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-kube-api-access-7ftf4\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:06.920128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.920076 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-oauth-config\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:06.920319 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:06.920135 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-service-ca\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.021562 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.021526 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-config\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.021859 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.021582 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-oauth-serving-cert\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.021859 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.021616 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-trusted-ca-bundle\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.021859 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.021636 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ftf4\" (UniqueName: \"kubernetes.io/projected/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-kube-api-access-7ftf4\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.021859 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.021666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-oauth-config\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.021859 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.021699 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-service-ca\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.021859 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.021728 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-serving-cert\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.022442 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.022405 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-config\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.022721 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.022696 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-service-ca\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.023068 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.023048 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-oauth-serving-cert\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.023312 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.023289 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-trusted-ca-bundle\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.024763 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.024737 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-oauth-config\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.024958 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.024936 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-serving-cert\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.030833 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.030809 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ftf4\" (UniqueName: \"kubernetes.io/projected/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-kube-api-access-7ftf4\") pod \"console-7bb8985c9d-gz5tb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:07.154524 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:07.154485 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:09.414257 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:09.414233 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bb8985c9d-gz5tb"] Jul 17 01:15:09.416704 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:15:09.416681 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2daf50_ae1c_45b0_8b26_ef781005c6fb.slice/crio-688ec20295fdf9dfbf65a9fe1002cf71e04c4c8dfe724e3b39441d4cf4eab7f7 WatchSource:0}: Error finding container 688ec20295fdf9dfbf65a9fe1002cf71e04c4c8dfe724e3b39441d4cf4eab7f7: Status 404 returned error can't find the container with id 688ec20295fdf9dfbf65a9fe1002cf71e04c4c8dfe724e3b39441d4cf4eab7f7 Jul 17 01:15:09.580612 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:09.580576 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zgf6x" event={"ID":"a5f557eb-8315-4fb5-803c-9d0c7961cb86","Type":"ContainerStarted","Data":"63ec45eb45ab77917b14e4942fb0d38cce804e3a9230af5d32ec9576c83d4d7b"} Jul 17 01:15:09.582272 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:09.582239 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-64dhp" event={"ID":"d7ed04a1-66ad-456b-93b4-1ddf01000212","Type":"ContainerStarted","Data":"7f79b262b54f48d66c62346fd6ab49b649d35411052365c0ba0e9fe12db3bea8"} Jul 17 01:15:09.582526 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:09.582347 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:15:09.583947 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:09.583922 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb8985c9d-gz5tb" event={"ID":"2e2daf50-ae1c-45b0-8b26-ef781005c6fb","Type":"ContainerStarted","Data":"dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc"} Jul 17 01:15:09.584066 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:09.583965 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb8985c9d-gz5tb" event={"ID":"2e2daf50-ae1c-45b0-8b26-ef781005c6fb","Type":"ContainerStarted","Data":"688ec20295fdf9dfbf65a9fe1002cf71e04c4c8dfe724e3b39441d4cf4eab7f7"} Jul 17 01:15:09.596864 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:09.596818 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zgf6x" podStartSLOduration=66.604868377 podStartE2EDuration="1m15.596801784s" totalImagesPullingTime="8.991933407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:15:09.59510316 +0000 UTC m=+75.973027569" watchObservedRunningTime="2026-07-17 01:15:09.596801784 +0000 UTC m=+75.974726192" Jul 17 01:15:09.609656 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:09.609606 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-64dhp" podStartSLOduration=67.360862988 podStartE2EDuration="1m15.609591224s" totalImagesPullingTime="8.248728236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:15:09.608819952 +0000 UTC m=+75.986744360" watchObservedRunningTime="2026-07-17 01:15:09.609591224 +0000 UTC m=+75.987515632" Jul 17 01:15:09.627088 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:09.627037 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bb8985c9d-gz5tb" podStartSLOduration=3.627023818 podStartE2EDuration="3.627023818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:15:09.625991871 +0000 UTC m=+76.003916279" watchObservedRunningTime="2026-07-17 01:15:09.627023818 +0000 UTC m=+76.004948294" Jul 17 01:15:17.155065 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:17.154942 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:17.155065 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:17.155043 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:17.159940 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:17.159914 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:17.618295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:17.618224 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:15:17.661802 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:17.661769 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b7fff644c-6p5xn"] Jul 17 01:15:29.634421 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.634377 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6696745474-9gcpg" podUID="bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" containerName="console" containerID="cri-o://b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae" gracePeriod=15 Jul 17 01:15:29.895285 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.895218 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:15:29.905506 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.905480 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm5dd\" (UniqueName: \"kubernetes.io/projected/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-kube-api-access-dm5dd\") pod \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " Jul 17 01:15:29.905649 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.905524 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-service-ca\") pod \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " Jul 17 01:15:29.905649 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.905544 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-config\") pod \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " Jul 17 01:15:29.905649 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.905598 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-oauth-serving-cert\") pod \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " Jul 17 01:15:29.905957 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.905932 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-config" (OuterVolumeSpecName: "console-config") pod "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" (UID: "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:15:29.905957 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.905941 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-service-ca" (OuterVolumeSpecName: "service-ca") pod "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" (UID: "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:15:29.906081 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.905985 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" (UID: "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:15:29.907981 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:29.907958 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-kube-api-access-dm5dd" (OuterVolumeSpecName: "kube-api-access-dm5dd") pod "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" (UID: "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354"). InnerVolumeSpecName "kube-api-access-dm5dd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:15:30.006563 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.006529 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-serving-cert\") pod \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " Jul 17 01:15:30.006563 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.006567 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-oauth-config\") pod \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\" (UID: \"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354\") " Jul 17 01:15:30.006774 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.006695 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-service-ca\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:30.006774 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.006705 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-config\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:30.006774 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.006713 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-oauth-serving-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:30.006774 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.006722 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm5dd\" (UniqueName: \"kubernetes.io/projected/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-kube-api-access-dm5dd\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:30.008775 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.008739 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" (UID: "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:15:30.008888 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.008819 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" (UID: "bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:15:30.107406 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.107371 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-serving-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:30.107406 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.107401 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354-console-oauth-config\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:30.661179 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.661127 2564 generic.go:356] "Generic (PLEG): container finished" podID="bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" containerID="b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae" exitCode=0 Jul 17 01:15:30.661689 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.661204 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6696745474-9gcpg" Jul 17 01:15:30.661689 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.661217 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6696745474-9gcpg" event={"ID":"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354","Type":"ContainerDied","Data":"b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae"} Jul 17 01:15:30.661689 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.661274 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6696745474-9gcpg" event={"ID":"bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354","Type":"ContainerDied","Data":"bee1fefe762e00bb09951d6b80db568c6a2de0668db63cad1b9ce784b1fabc5f"} Jul 17 01:15:30.661689 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.661295 2564 scope.go:117] "RemoveContainer" containerID="b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae" Jul 17 01:15:30.669546 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.669523 2564 scope.go:117] "RemoveContainer" containerID="b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae" Jul 17 01:15:30.669846 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:15:30.669825 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae\": container with ID starting with b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae not found: ID does not exist" containerID="b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae" Jul 17 01:15:30.669927 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.669855 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae"} err="failed to get container status \"b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae\": rpc error: code = NotFound desc = could not find container \"b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae\": container with ID starting with b0eec6ce64ab243b3ba2aca87bff4a0c325842dbf51083765a0a0b0c16672dae not found: ID does not exist" Jul 17 01:15:30.676240 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.676218 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6696745474-9gcpg"] Jul 17 01:15:30.679648 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:30.679629 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6696745474-9gcpg"] Jul 17 01:15:32.133696 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:32.133653 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" path="/var/lib/kubelet/pods/bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354/volumes" Jul 17 01:15:34.802079 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.802043 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d47664467-9h5tk"] Jul 17 01:15:34.802473 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.802307 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" containerName="console" Jul 17 01:15:34.802473 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.802318 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" containerName="console" Jul 17 01:15:34.802473 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.802358 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bab2a9ea-cab5-4aac-9dd8-1a1eea3f6354" containerName="console" Jul 17 01:15:34.863841 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.863796 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d47664467-9h5tk"] Jul 17 01:15:34.864005 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.863922 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:34.940952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.940910 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-serving-cert\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:34.940952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.940947 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-service-ca\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:34.941155 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.940975 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-oauth-serving-cert\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:34.941155 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.941031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-oauth-config\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:34.941155 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.941062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-trusted-ca-bundle\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:34.941155 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.941120 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ck9r\" (UniqueName: \"kubernetes.io/projected/c2467b83-14db-4717-9e1c-3a1fb98dd39d-kube-api-access-8ck9r\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:34.941155 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:34.941147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-config\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.042182 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.042139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-oauth-config\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.042366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.042206 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-trusted-ca-bundle\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.042366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.042234 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ck9r\" (UniqueName: \"kubernetes.io/projected/c2467b83-14db-4717-9e1c-3a1fb98dd39d-kube-api-access-8ck9r\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.042366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.042257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-config\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.042366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.042289 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-serving-cert\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.042366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.042314 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-service-ca\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.042366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.042355 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-oauth-serving-cert\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.042986 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.042959 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-config\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.043264 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.043245 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-trusted-ca-bundle\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.043331 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.043308 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-oauth-serving-cert\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.043453 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.043431 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-service-ca\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.044947 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.044921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-serving-cert\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.045061 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.044953 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-oauth-config\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.050662 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.050640 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ck9r\" (UniqueName: \"kubernetes.io/projected/c2467b83-14db-4717-9e1c-3a1fb98dd39d-kube-api-access-8ck9r\") pod \"console-d47664467-9h5tk\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.172886 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.172846 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:35.331066 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.331037 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d47664467-9h5tk"] Jul 17 01:15:35.333819 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:15:35.333792 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2467b83_14db_4717_9e1c_3a1fb98dd39d.slice/crio-eb38b6858f2a2cbababdc193b14002ad878fafde92f4766d4cf528cfc3d051be WatchSource:0}: Error finding container eb38b6858f2a2cbababdc193b14002ad878fafde92f4766d4cf528cfc3d051be: Status 404 returned error can't find the container with id eb38b6858f2a2cbababdc193b14002ad878fafde92f4766d4cf528cfc3d051be Jul 17 01:15:35.682423 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.682387 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d47664467-9h5tk" event={"ID":"c2467b83-14db-4717-9e1c-3a1fb98dd39d","Type":"ContainerStarted","Data":"895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456"} Jul 17 01:15:35.682602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.682442 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d47664467-9h5tk" event={"ID":"c2467b83-14db-4717-9e1c-3a1fb98dd39d","Type":"ContainerStarted","Data":"eb38b6858f2a2cbababdc193b14002ad878fafde92f4766d4cf528cfc3d051be"} Jul 17 01:15:35.700685 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:35.700588 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d47664467-9h5tk" podStartSLOduration=1.7005710760000001 podStartE2EDuration="1.700571076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:15:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:15:35.699376073 +0000 UTC m=+102.077300480" watchObservedRunningTime="2026-07-17 01:15:35.700571076 +0000 UTC m=+102.078495483" Jul 17 01:15:36.782970 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.782938 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 01:15:36.807116 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.807082 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 01:15:36.807281 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.807249 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.810427 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.810399 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 17 01:15:36.810573 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.810435 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 17 01:15:36.810573 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.810461 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 17 01:15:36.810573 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.810407 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 17 01:15:36.810573 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.810495 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 17 01:15:36.810573 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.810465 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 17 01:15:36.810823 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.810659 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zdbz6\"" Jul 17 01:15:36.810823 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.810722 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 17 01:15:36.810963 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.810942 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 17 01:15:36.815491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.815469 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 17 01:15:36.852734 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.852702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6807f673-2152-4c92-af6f-ad54a23ab100-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.852734 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.852735 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.852952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.852754 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.852952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.852770 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.852952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.852834 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6807f673-2152-4c92-af6f-ad54a23ab100-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.852952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.852860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-config-volume\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.852952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.852918 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-web-config\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.853150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.852959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqwhv\" (UniqueName: \"kubernetes.io/projected/6807f673-2152-4c92-af6f-ad54a23ab100-kube-api-access-vqwhv\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.853150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.852981 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6807f673-2152-4c92-af6f-ad54a23ab100-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.853150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.853011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6807f673-2152-4c92-af6f-ad54a23ab100-config-out\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.853150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.853027 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.853150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.853051 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6807f673-2152-4c92-af6f-ad54a23ab100-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.853150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.853067 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.953745 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.953708 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6807f673-2152-4c92-af6f-ad54a23ab100-config-out\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.953745 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.953744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.953975 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.953766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6807f673-2152-4c92-af6f-ad54a23ab100-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.953975 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.953879 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.953975 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.953912 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6807f673-2152-4c92-af6f-ad54a23ab100-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.953975 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.953937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.953975 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.953967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.954257 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.953995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.954257 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.954046 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6807f673-2152-4c92-af6f-ad54a23ab100-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.954257 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.954085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-config-volume\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.954257 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.954126 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-web-config\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.954257 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.954179 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6807f673-2152-4c92-af6f-ad54a23ab100-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.954257 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.954204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqwhv\" (UniqueName: \"kubernetes.io/projected/6807f673-2152-4c92-af6f-ad54a23ab100-kube-api-access-vqwhv\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.954257 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.954235 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6807f673-2152-4c92-af6f-ad54a23ab100-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.955139 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.955112 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6807f673-2152-4c92-af6f-ad54a23ab100-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.957122 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.957094 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.957265 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.957104 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.957328 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.957262 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-config-volume\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.957328 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.957277 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.957975 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.957953 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-web-config\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.958080 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.958031 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6807f673-2152-4c92-af6f-ad54a23ab100-config-out\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.958142 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.958106 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6807f673-2152-4c92-af6f-ad54a23ab100-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.958457 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.958424 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.959427 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.959404 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6807f673-2152-4c92-af6f-ad54a23ab100-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.960045 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.960024 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6807f673-2152-4c92-af6f-ad54a23ab100-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:36.965298 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:36.965273 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqwhv\" (UniqueName: \"kubernetes.io/projected/6807f673-2152-4c92-af6f-ad54a23ab100-kube-api-access-vqwhv\") pod \"alertmanager-main-0\" (UID: \"6807f673-2152-4c92-af6f-ad54a23ab100\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:37.117231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:37.117133 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 01:15:37.294835 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:37.294801 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 01:15:37.298363 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:15:37.298330 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6807f673_2152_4c92_af6f_ad54a23ab100.slice/crio-acc12f2f23c1d294a2cff7c97aa6c28ddfb6d4ad6a15e1f9d39c86f15fe172bc WatchSource:0}: Error finding container acc12f2f23c1d294a2cff7c97aa6c28ddfb6d4ad6a15e1f9d39c86f15fe172bc: Status 404 returned error can't find the container with id acc12f2f23c1d294a2cff7c97aa6c28ddfb6d4ad6a15e1f9d39c86f15fe172bc Jul 17 01:15:37.692045 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:37.692001 2564 generic.go:356] "Generic (PLEG): container finished" podID="6807f673-2152-4c92-af6f-ad54a23ab100" containerID="cc0b0d069320bc8b6db2aa5d6c40c307fe2b86b9dc15b5e141b5bd7855a72920" exitCode=0 Jul 17 01:15:37.692223 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:37.692095 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6807f673-2152-4c92-af6f-ad54a23ab100","Type":"ContainerDied","Data":"cc0b0d069320bc8b6db2aa5d6c40c307fe2b86b9dc15b5e141b5bd7855a72920"} Jul 17 01:15:37.692223 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:37.692202 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6807f673-2152-4c92-af6f-ad54a23ab100","Type":"ContainerStarted","Data":"acc12f2f23c1d294a2cff7c97aa6c28ddfb6d4ad6a15e1f9d39c86f15fe172bc"} Jul 17 01:15:40.590211 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:40.590157 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-64dhp" Jul 17 01:15:40.708597 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:40.708566 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6807f673-2152-4c92-af6f-ad54a23ab100","Type":"ContainerStarted","Data":"ed418bad465523db929083466acab8a458ad08ca92a9c16cab4ff14626876b13"} Jul 17 01:15:40.708778 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:40.708630 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6807f673-2152-4c92-af6f-ad54a23ab100","Type":"ContainerStarted","Data":"761527cfdd2590ec56a76591aaa5f2b81bbc01faf825f8c97dbc21f0f3d44254"} Jul 17 01:15:40.708778 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:40.708662 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6807f673-2152-4c92-af6f-ad54a23ab100","Type":"ContainerStarted","Data":"bffbb5a64da7840be785f6f24e4e91fa937b6b130cc41187604a09c8dc78284a"} Jul 17 01:15:41.715086 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:41.715055 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6807f673-2152-4c92-af6f-ad54a23ab100","Type":"ContainerStarted","Data":"285f04c805e666e7d973bd9922c61bb59d5850f7b3f92edd073f0833b658e1b8"} Jul 17 01:15:41.715528 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:41.715120 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6807f673-2152-4c92-af6f-ad54a23ab100","Type":"ContainerStarted","Data":"3f057ab484ea3c78d6839ca163b3786fab91fb179d2c329a2a61eeaee3d49ebb"} Jul 17 01:15:42.688614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:42.688571 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b7fff644c-6p5xn" podUID="c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" containerName="console" containerID="cri-o://7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c" gracePeriod=15 Jul 17 01:15:42.723338 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:42.723311 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6807f673-2152-4c92-af6f-ad54a23ab100","Type":"ContainerStarted","Data":"e3858f8eb5dd3f00b1067034bde2f41fb8bf3c50b636df13c7e3da9556723e60"} Jul 17 01:15:42.748671 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:42.748619 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.972286619 podStartE2EDuration="6.748602869s" totalImagesPullingTime="3.754666343s" totalInitContainerRuntime="21.649907ms" isStatefulPod=false podCreationTimestamp="2026-07-17 01:15:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:15:42.746676739 +0000 UTC m=+109.124601170" watchObservedRunningTime="2026-07-17 01:15:42.748602869 +0000 UTC m=+109.126527277" Jul 17 01:15:42.960926 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:42.960862 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:15:43.012198 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012148 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grvh2\" (UniqueName: \"kubernetes.io/projected/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-kube-api-access-grvh2\") pod \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " Jul 17 01:15:43.012359 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012214 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-oauth-config\") pod \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " Jul 17 01:15:43.012359 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012240 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-oauth-serving-cert\") pod \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " Jul 17 01:15:43.012443 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012350 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-trusted-ca-bundle\") pod \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " Jul 17 01:15:43.012443 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012403 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-service-ca\") pod \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " Jul 17 01:15:43.012443 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012433 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-serving-cert\") pod \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " Jul 17 01:15:43.012559 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012473 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-config\") pod \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\" (UID: \"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1\") " Jul 17 01:15:43.012666 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012642 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" (UID: "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:15:43.012763 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012739 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-service-ca" (OuterVolumeSpecName: "service-ca") pod "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" (UID: "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:15:43.013041 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012748 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-oauth-serving-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:43.013041 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012756 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" (UID: "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:15:43.013041 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.012992 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-config" (OuterVolumeSpecName: "console-config") pod "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" (UID: "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:15:43.014497 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.014476 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-kube-api-access-grvh2" (OuterVolumeSpecName: "kube-api-access-grvh2") pod "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" (UID: "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1"). InnerVolumeSpecName "kube-api-access-grvh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:15:43.015017 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.015001 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" (UID: "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:15:43.015098 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.015077 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" (UID: "c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:15:43.113827 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.113790 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grvh2\" (UniqueName: \"kubernetes.io/projected/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-kube-api-access-grvh2\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:43.113827 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.113823 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-oauth-config\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:43.113827 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.113832 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-trusted-ca-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:43.114053 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.113841 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-service-ca\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:43.114053 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.113851 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-serving-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:43.114053 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.113859 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1-console-config\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:15:43.728020 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.727983 2564 generic.go:356] "Generic (PLEG): container finished" podID="c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" containerID="7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c" exitCode=0 Jul 17 01:15:43.728445 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.728029 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7fff644c-6p5xn" Jul 17 01:15:43.728445 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.728077 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7fff644c-6p5xn" event={"ID":"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1","Type":"ContainerDied","Data":"7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c"} Jul 17 01:15:43.728445 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.728135 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7fff644c-6p5xn" event={"ID":"c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1","Type":"ContainerDied","Data":"5b694102f3abb76e73f8c13243df18c7275cf1a54c7a2aa1d3d84d62ca016006"} Jul 17 01:15:43.728445 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.728177 2564 scope.go:117] "RemoveContainer" containerID="7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c" Jul 17 01:15:43.740695 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.740678 2564 scope.go:117] "RemoveContainer" containerID="7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c" Jul 17 01:15:43.740966 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:15:43.740945 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c\": container with ID starting with 7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c not found: ID does not exist" containerID="7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c" Jul 17 01:15:43.741060 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.740970 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c"} err="failed to get container status \"7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c\": rpc error: code = NotFound desc = could not find container \"7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c\": container with ID starting with 7bf10e438b7287d9bd21767d37a80e49a448f6f68228019f5953343611734d6c not found: ID does not exist" Jul 17 01:15:43.748463 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.748436 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b7fff644c-6p5xn"] Jul 17 01:15:43.753763 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:43.753741 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b7fff644c-6p5xn"] Jul 17 01:15:44.132958 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:44.132887 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" path="/var/lib/kubelet/pods/c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1/volumes" Jul 17 01:15:45.173704 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:45.173667 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:45.174114 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:45.173737 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:45.178581 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:45.178555 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:45.740681 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:45.740653 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:15:45.785074 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:15:45.785038 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bb8985c9d-gz5tb"] Jul 17 01:16:10.805536 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:10.805470 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bb8985c9d-gz5tb" podUID="2e2daf50-ae1c-45b0-8b26-ef781005c6fb" containerName="console" containerID="cri-o://dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc" gracePeriod=15 Jul 17 01:16:11.038938 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.038913 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:16:11.143214 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143105 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-oauth-serving-cert\") pod \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " Jul 17 01:16:11.143214 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143157 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-trusted-ca-bundle\") pod \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " Jul 17 01:16:11.143214 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143215 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-oauth-config\") pod \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " Jul 17 01:16:11.143482 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143309 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-serving-cert\") pod \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " Jul 17 01:16:11.143482 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143373 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-service-ca\") pod \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " Jul 17 01:16:11.143482 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143412 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-config\") pod \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " Jul 17 01:16:11.143482 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143452 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ftf4\" (UniqueName: \"kubernetes.io/projected/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-kube-api-access-7ftf4\") pod \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\" (UID: \"2e2daf50-ae1c-45b0-8b26-ef781005c6fb\") " Jul 17 01:16:11.143681 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143615 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2e2daf50-ae1c-45b0-8b26-ef781005c6fb" (UID: "2e2daf50-ae1c-45b0-8b26-ef781005c6fb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:16:11.143681 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143670 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2e2daf50-ae1c-45b0-8b26-ef781005c6fb" (UID: "2e2daf50-ae1c-45b0-8b26-ef781005c6fb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:16:11.143804 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143770 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-service-ca" (OuterVolumeSpecName: "service-ca") pod "2e2daf50-ae1c-45b0-8b26-ef781005c6fb" (UID: "2e2daf50-ae1c-45b0-8b26-ef781005c6fb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:16:11.143865 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143801 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-oauth-serving-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:11.143865 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143823 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-trusted-ca-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:11.143865 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.143827 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-config" (OuterVolumeSpecName: "console-config") pod "2e2daf50-ae1c-45b0-8b26-ef781005c6fb" (UID: "2e2daf50-ae1c-45b0-8b26-ef781005c6fb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:16:11.145575 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.145547 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2e2daf50-ae1c-45b0-8b26-ef781005c6fb" (UID: "2e2daf50-ae1c-45b0-8b26-ef781005c6fb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:16:11.145726 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.145704 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-kube-api-access-7ftf4" (OuterVolumeSpecName: "kube-api-access-7ftf4") pod "2e2daf50-ae1c-45b0-8b26-ef781005c6fb" (UID: "2e2daf50-ae1c-45b0-8b26-ef781005c6fb"). InnerVolumeSpecName "kube-api-access-7ftf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:16:11.145808 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.145785 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2e2daf50-ae1c-45b0-8b26-ef781005c6fb" (UID: "2e2daf50-ae1c-45b0-8b26-ef781005c6fb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:16:11.244619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.244570 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-config\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:11.244619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.244611 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ftf4\" (UniqueName: \"kubernetes.io/projected/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-kube-api-access-7ftf4\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:11.244619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.244622 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-oauth-config\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:11.244619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.244631 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-console-serving-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:11.244619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.244639 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2e2daf50-ae1c-45b0-8b26-ef781005c6fb-service-ca\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:11.833497 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.833462 2564 generic.go:356] "Generic (PLEG): container finished" podID="2e2daf50-ae1c-45b0-8b26-ef781005c6fb" containerID="dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc" exitCode=0 Jul 17 01:16:11.833885 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.833512 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bb8985c9d-gz5tb" Jul 17 01:16:11.833885 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.833548 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb8985c9d-gz5tb" event={"ID":"2e2daf50-ae1c-45b0-8b26-ef781005c6fb","Type":"ContainerDied","Data":"dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc"} Jul 17 01:16:11.833885 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.833602 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bb8985c9d-gz5tb" event={"ID":"2e2daf50-ae1c-45b0-8b26-ef781005c6fb","Type":"ContainerDied","Data":"688ec20295fdf9dfbf65a9fe1002cf71e04c4c8dfe724e3b39441d4cf4eab7f7"} Jul 17 01:16:11.833885 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.833623 2564 scope.go:117] "RemoveContainer" containerID="dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc" Jul 17 01:16:11.858831 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.858202 2564 scope.go:117] "RemoveContainer" containerID="dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc" Jul 17 01:16:11.858831 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:16:11.858582 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc\": container with ID starting with dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc not found: ID does not exist" containerID="dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc" Jul 17 01:16:11.858831 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.858615 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc"} err="failed to get container status \"dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc\": rpc error: code = NotFound desc = could not find container \"dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc\": container with ID starting with dca5bdfda64acb4c69bca82281f52a04863257819f7a8f4d7cde7aa6c385effc not found: ID does not exist" Jul 17 01:16:11.873745 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.873718 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bb8985c9d-gz5tb"] Jul 17 01:16:11.876599 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:11.876576 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bb8985c9d-gz5tb"] Jul 17 01:16:12.132479 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:12.132405 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e2daf50-ae1c-45b0-8b26-ef781005c6fb" path="/var/lib/kubelet/pods/2e2daf50-ae1c-45b0-8b26-ef781005c6fb/volumes" Jul 17 01:16:33.209832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.209800 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz"] Jul 17 01:16:33.210307 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.210090 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" containerName="console" Jul 17 01:16:33.210307 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.210102 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" containerName="console" Jul 17 01:16:33.210307 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.210115 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e2daf50-ae1c-45b0-8b26-ef781005c6fb" containerName="console" Jul 17 01:16:33.210307 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.210121 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e2daf50-ae1c-45b0-8b26-ef781005c6fb" containerName="console" Jul 17 01:16:33.210307 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.210189 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e2daf50-ae1c-45b0-8b26-ef781005c6fb" containerName="console" Jul 17 01:16:33.210307 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.210197 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6fbb12c-3ee7-4dcb-82e1-384f3ec38ba1" containerName="console" Jul 17 01:16:33.212024 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.212008 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.214665 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.214646 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 01:16:33.214665 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.214648 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 01:16:33.215760 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.215735 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sgd4t\"" Jul 17 01:16:33.221432 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.221411 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz"] Jul 17 01:16:33.224813 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.224784 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpcsd\" (UniqueName: \"kubernetes.io/projected/94970392-f48a-4b4d-a072-79d0d956caa8-kube-api-access-bpcsd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.224927 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.224833 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.224997 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.224934 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.325655 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.325613 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.325655 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.325675 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcsd\" (UniqueName: \"kubernetes.io/projected/94970392-f48a-4b4d-a072-79d0d956caa8-kube-api-access-bpcsd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.325655 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.325701 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.326114 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.326023 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.326114 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.326041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.337280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.337247 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpcsd\" (UniqueName: \"kubernetes.io/projected/94970392-f48a-4b4d-a072-79d0d956caa8-kube-api-access-bpcsd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.522345 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.522253 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:33.648572 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.648429 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz"] Jul 17 01:16:33.651065 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:16:33.651041 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94970392_f48a_4b4d_a072_79d0d956caa8.slice/crio-4f73f5ba9447862a5ac160c1df6f7ea076000204431bd4691e6c950ae3cb3d02 WatchSource:0}: Error finding container 4f73f5ba9447862a5ac160c1df6f7ea076000204431bd4691e6c950ae3cb3d02: Status 404 returned error can't find the container with id 4f73f5ba9447862a5ac160c1df6f7ea076000204431bd4691e6c950ae3cb3d02 Jul 17 01:16:33.914969 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:33.914937 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" event={"ID":"94970392-f48a-4b4d-a072-79d0d956caa8","Type":"ContainerStarted","Data":"4f73f5ba9447862a5ac160c1df6f7ea076000204431bd4691e6c950ae3cb3d02"} Jul 17 01:16:40.945726 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:40.945686 2564 generic.go:356] "Generic (PLEG): container finished" podID="94970392-f48a-4b4d-a072-79d0d956caa8" containerID="abd87641f3cb44acdf466e258d78423feb697e45a45f15a868fe092dd8138fc1" exitCode=0 Jul 17 01:16:40.946100 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:40.945752 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" event={"ID":"94970392-f48a-4b4d-a072-79d0d956caa8","Type":"ContainerDied","Data":"abd87641f3cb44acdf466e258d78423feb697e45a45f15a868fe092dd8138fc1"} Jul 17 01:16:43.960995 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:43.960957 2564 generic.go:356] "Generic (PLEG): container finished" podID="94970392-f48a-4b4d-a072-79d0d956caa8" containerID="88b386c1dee1377dc26dc1dd612618e6d70f2a0ecf5e79a73faacc922d3aee22" exitCode=0 Jul 17 01:16:43.961402 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:43.961038 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" event={"ID":"94970392-f48a-4b4d-a072-79d0d956caa8","Type":"ContainerDied","Data":"88b386c1dee1377dc26dc1dd612618e6d70f2a0ecf5e79a73faacc922d3aee22"} Jul 17 01:16:51.994487 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:51.994453 2564 generic.go:356] "Generic (PLEG): container finished" podID="94970392-f48a-4b4d-a072-79d0d956caa8" containerID="09c023e1a2f6691add3b814abcb8d2afbd5b20864b1bd527cb1800411a0771f7" exitCode=0 Jul 17 01:16:51.994857 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:51.994522 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" event={"ID":"94970392-f48a-4b4d-a072-79d0d956caa8","Type":"ContainerDied","Data":"09c023e1a2f6691add3b814abcb8d2afbd5b20864b1bd527cb1800411a0771f7"} Jul 17 01:16:53.128284 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.128258 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:53.156681 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.156649 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-bundle\") pod \"94970392-f48a-4b4d-a072-79d0d956caa8\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " Jul 17 01:16:53.156868 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.156737 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-util\") pod \"94970392-f48a-4b4d-a072-79d0d956caa8\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " Jul 17 01:16:53.156868 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.156771 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bpcsd\" (UniqueName: \"kubernetes.io/projected/94970392-f48a-4b4d-a072-79d0d956caa8-kube-api-access-bpcsd\") pod \"94970392-f48a-4b4d-a072-79d0d956caa8\" (UID: \"94970392-f48a-4b4d-a072-79d0d956caa8\") " Jul 17 01:16:53.157356 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.157331 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-bundle" (OuterVolumeSpecName: "bundle") pod "94970392-f48a-4b4d-a072-79d0d956caa8" (UID: "94970392-f48a-4b4d-a072-79d0d956caa8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:16:53.159194 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.159143 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94970392-f48a-4b4d-a072-79d0d956caa8-kube-api-access-bpcsd" (OuterVolumeSpecName: "kube-api-access-bpcsd") pod "94970392-f48a-4b4d-a072-79d0d956caa8" (UID: "94970392-f48a-4b4d-a072-79d0d956caa8"). InnerVolumeSpecName "kube-api-access-bpcsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:16:53.161520 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.161493 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-util" (OuterVolumeSpecName: "util") pod "94970392-f48a-4b4d-a072-79d0d956caa8" (UID: "94970392-f48a-4b4d-a072-79d0d956caa8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:16:53.258383 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.258295 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-util\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:53.258383 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.258325 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bpcsd\" (UniqueName: \"kubernetes.io/projected/94970392-f48a-4b4d-a072-79d0d956caa8-kube-api-access-bpcsd\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:53.258383 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:53.258335 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/94970392-f48a-4b4d-a072-79d0d956caa8-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:16:54.004136 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:54.004107 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" event={"ID":"94970392-f48a-4b4d-a072-79d0d956caa8","Type":"ContainerDied","Data":"4f73f5ba9447862a5ac160c1df6f7ea076000204431bd4691e6c950ae3cb3d02"} Jul 17 01:16:54.004323 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:54.004142 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rc6vz" Jul 17 01:16:54.004323 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:16:54.004153 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f73f5ba9447862a5ac160c1df6f7ea076000204431bd4691e6c950ae3cb3d02" Jul 17 01:17:01.122815 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.122778 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb"] Jul 17 01:17:01.123292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.123033 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94970392-f48a-4b4d-a072-79d0d956caa8" containerName="extract" Jul 17 01:17:01.123292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.123043 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="94970392-f48a-4b4d-a072-79d0d956caa8" containerName="extract" Jul 17 01:17:01.123292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.123054 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94970392-f48a-4b4d-a072-79d0d956caa8" containerName="pull" Jul 17 01:17:01.123292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.123059 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="94970392-f48a-4b4d-a072-79d0d956caa8" containerName="pull" Jul 17 01:17:01.123292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.123068 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94970392-f48a-4b4d-a072-79d0d956caa8" containerName="util" Jul 17 01:17:01.123292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.123073 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="94970392-f48a-4b4d-a072-79d0d956caa8" containerName="util" Jul 17 01:17:01.123292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.123125 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="94970392-f48a-4b4d-a072-79d0d956caa8" containerName="extract" Jul 17 01:17:01.176916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.176876 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb"] Jul 17 01:17:01.177072 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.177000 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" Jul 17 01:17:01.179993 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.179965 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qzckk\"" Jul 17 01:17:01.179993 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.179966 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 17 01:17:01.180224 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.180006 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 17 01:17:01.216049 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.216011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66639ff6-cea8-4ef1-a9d2-80f14b9182d6-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x6qb\" (UID: \"66639ff6-cea8-4ef1-a9d2-80f14b9182d6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" Jul 17 01:17:01.216049 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.216048 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfdr5\" (UniqueName: \"kubernetes.io/projected/66639ff6-cea8-4ef1-a9d2-80f14b9182d6-kube-api-access-jfdr5\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x6qb\" (UID: \"66639ff6-cea8-4ef1-a9d2-80f14b9182d6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" Jul 17 01:17:01.316994 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.316950 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66639ff6-cea8-4ef1-a9d2-80f14b9182d6-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x6qb\" (UID: \"66639ff6-cea8-4ef1-a9d2-80f14b9182d6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" Jul 17 01:17:01.316994 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.316997 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfdr5\" (UniqueName: \"kubernetes.io/projected/66639ff6-cea8-4ef1-a9d2-80f14b9182d6-kube-api-access-jfdr5\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x6qb\" (UID: \"66639ff6-cea8-4ef1-a9d2-80f14b9182d6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" Jul 17 01:17:01.317433 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.317413 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66639ff6-cea8-4ef1-a9d2-80f14b9182d6-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x6qb\" (UID: \"66639ff6-cea8-4ef1-a9d2-80f14b9182d6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" Jul 17 01:17:01.327655 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.327624 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfdr5\" (UniqueName: \"kubernetes.io/projected/66639ff6-cea8-4ef1-a9d2-80f14b9182d6-kube-api-access-jfdr5\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x6qb\" (UID: \"66639ff6-cea8-4ef1-a9d2-80f14b9182d6\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" Jul 17 01:17:01.485802 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.485767 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" Jul 17 01:17:01.640679 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:01.640653 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb"] Jul 17 01:17:01.643524 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:17:01.643482 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66639ff6_cea8_4ef1_a9d2_80f14b9182d6.slice/crio-5a9ca35d19980639cdeaeadd535367cd6e1849e9be0897c26db808567f9ca8fa WatchSource:0}: Error finding container 5a9ca35d19980639cdeaeadd535367cd6e1849e9be0897c26db808567f9ca8fa: Status 404 returned error can't find the container with id 5a9ca35d19980639cdeaeadd535367cd6e1849e9be0897c26db808567f9ca8fa Jul 17 01:17:02.033328 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:02.033290 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" event={"ID":"66639ff6-cea8-4ef1-a9d2-80f14b9182d6","Type":"ContainerStarted","Data":"5a9ca35d19980639cdeaeadd535367cd6e1849e9be0897c26db808567f9ca8fa"} Jul 17 01:17:04.044475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:04.044387 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" event={"ID":"66639ff6-cea8-4ef1-a9d2-80f14b9182d6","Type":"ContainerStarted","Data":"73723103559b107c0a71082640ba0fa4e6b88cea6851d85b6e4c1ed5f7e88c10"} Jul 17 01:17:04.066324 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:04.066272 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x6qb" podStartSLOduration=0.983518215 podStartE2EDuration="3.066257229s" totalImagesPullingTime="2.082739014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:17:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:17:04.064592727 +0000 UTC m=+190.442517143" watchObservedRunningTime="2026-07-17 01:17:04.066257229 +0000 UTC m=+190.444181638" Jul 17 01:17:05.258223 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.258180 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww"] Jul 17 01:17:05.260587 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.260563 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.263157 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.263133 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 01:17:05.263318 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.263242 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 01:17:05.264273 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.264249 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sgd4t\"" Jul 17 01:17:05.270257 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.270228 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww"] Jul 17 01:17:05.349496 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.349454 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.349496 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.349492 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sh5d\" (UniqueName: \"kubernetes.io/projected/60610957-4a9a-4a48-be3c-552281770c88-kube-api-access-9sh5d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.349713 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.349528 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.450880 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.450843 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.450880 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.450879 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sh5d\" (UniqueName: \"kubernetes.io/projected/60610957-4a9a-4a48-be3c-552281770c88-kube-api-access-9sh5d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.451134 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.450906 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.451279 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.451261 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.451345 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.451323 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.462060 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.462033 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sh5d\" (UniqueName: \"kubernetes.io/projected/60610957-4a9a-4a48-be3c-552281770c88-kube-api-access-9sh5d\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.569197 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.569090 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:05.702088 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:05.702061 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww"] Jul 17 01:17:05.704909 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:17:05.704882 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60610957_4a9a_4a48_be3c_552281770c88.slice/crio-e4a2cce1389e760c2cd661ec4a03b707f72c28cc6c1572ffdb49f54be4c57c87 WatchSource:0}: Error finding container e4a2cce1389e760c2cd661ec4a03b707f72c28cc6c1572ffdb49f54be4c57c87: Status 404 returned error can't find the container with id e4a2cce1389e760c2cd661ec4a03b707f72c28cc6c1572ffdb49f54be4c57c87 Jul 17 01:17:06.054660 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:06.054626 2564 generic.go:356] "Generic (PLEG): container finished" podID="60610957-4a9a-4a48-be3c-552281770c88" containerID="0f7aa09e48205a98e3805d67c723085c29d69cd509417581579e39a650ee0bc4" exitCode=0 Jul 17 01:17:06.054847 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:06.054717 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" event={"ID":"60610957-4a9a-4a48-be3c-552281770c88","Type":"ContainerDied","Data":"0f7aa09e48205a98e3805d67c723085c29d69cd509417581579e39a650ee0bc4"} Jul 17 01:17:06.054847 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:06.054773 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" event={"ID":"60610957-4a9a-4a48-be3c-552281770c88","Type":"ContainerStarted","Data":"e4a2cce1389e760c2cd661ec4a03b707f72c28cc6c1572ffdb49f54be4c57c87"} Jul 17 01:17:07.742335 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.742296 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s"] Jul 17 01:17:07.749098 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.749065 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" Jul 17 01:17:07.752040 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.751999 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dghmt\"" Jul 17 01:17:07.752299 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.752287 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 17 01:17:07.753360 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.753188 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 17 01:17:07.755415 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.755390 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s"] Jul 17 01:17:07.872572 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.872531 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/468703cc-6053-40d8-a5ab-262253de2ff8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xtn7s\" (UID: \"468703cc-6053-40d8-a5ab-262253de2ff8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" Jul 17 01:17:07.872759 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.872618 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgdtm\" (UniqueName: \"kubernetes.io/projected/468703cc-6053-40d8-a5ab-262253de2ff8-kube-api-access-kgdtm\") pod \"cert-manager-cainjector-559cdd8c87-xtn7s\" (UID: \"468703cc-6053-40d8-a5ab-262253de2ff8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" Jul 17 01:17:07.974132 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.974098 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/468703cc-6053-40d8-a5ab-262253de2ff8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xtn7s\" (UID: \"468703cc-6053-40d8-a5ab-262253de2ff8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" Jul 17 01:17:07.974292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.974155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgdtm\" (UniqueName: \"kubernetes.io/projected/468703cc-6053-40d8-a5ab-262253de2ff8-kube-api-access-kgdtm\") pod \"cert-manager-cainjector-559cdd8c87-xtn7s\" (UID: \"468703cc-6053-40d8-a5ab-262253de2ff8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" Jul 17 01:17:07.982699 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.982676 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/468703cc-6053-40d8-a5ab-262253de2ff8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xtn7s\" (UID: \"468703cc-6053-40d8-a5ab-262253de2ff8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" Jul 17 01:17:07.982818 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:07.982800 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgdtm\" (UniqueName: \"kubernetes.io/projected/468703cc-6053-40d8-a5ab-262253de2ff8-kube-api-access-kgdtm\") pod \"cert-manager-cainjector-559cdd8c87-xtn7s\" (UID: \"468703cc-6053-40d8-a5ab-262253de2ff8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" Jul 17 01:17:08.065998 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:08.065966 2564 generic.go:356] "Generic (PLEG): container finished" podID="60610957-4a9a-4a48-be3c-552281770c88" containerID="90efd00ac51d501e9ac14a9cdd3c9df7093435938315dae6dccf1f29d8e61868" exitCode=0 Jul 17 01:17:08.065998 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:08.066003 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" event={"ID":"60610957-4a9a-4a48-be3c-552281770c88","Type":"ContainerDied","Data":"90efd00ac51d501e9ac14a9cdd3c9df7093435938315dae6dccf1f29d8e61868"} Jul 17 01:17:08.072446 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:08.072426 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" Jul 17 01:17:08.208537 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:08.208514 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s"] Jul 17 01:17:08.210751 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:17:08.210722 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod468703cc_6053_40d8_a5ab_262253de2ff8.slice/crio-7e6b28249e3de9dd9bd2e49dc401de57e96a6557c7517c5b7c4e09cc0150e530 WatchSource:0}: Error finding container 7e6b28249e3de9dd9bd2e49dc401de57e96a6557c7517c5b7c4e09cc0150e530: Status 404 returned error can't find the container with id 7e6b28249e3de9dd9bd2e49dc401de57e96a6557c7517c5b7c4e09cc0150e530 Jul 17 01:17:09.072185 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:09.072120 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" event={"ID":"468703cc-6053-40d8-a5ab-262253de2ff8","Type":"ContainerStarted","Data":"7e6b28249e3de9dd9bd2e49dc401de57e96a6557c7517c5b7c4e09cc0150e530"} Jul 17 01:17:09.078728 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:09.078698 2564 generic.go:356] "Generic (PLEG): container finished" podID="60610957-4a9a-4a48-be3c-552281770c88" containerID="b2719bcdee7acbb3513f54bd8400ac2844e23a76ccfd3795682672c0a97afa72" exitCode=0 Jul 17 01:17:09.078916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:09.078751 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" event={"ID":"60610957-4a9a-4a48-be3c-552281770c88","Type":"ContainerDied","Data":"b2719bcdee7acbb3513f54bd8400ac2844e23a76ccfd3795682672c0a97afa72"} Jul 17 01:17:10.821808 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:10.821784 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:10.899859 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:10.899834 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-util\") pod \"60610957-4a9a-4a48-be3c-552281770c88\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " Jul 17 01:17:10.900001 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:10.899875 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sh5d\" (UniqueName: \"kubernetes.io/projected/60610957-4a9a-4a48-be3c-552281770c88-kube-api-access-9sh5d\") pod \"60610957-4a9a-4a48-be3c-552281770c88\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " Jul 17 01:17:10.900001 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:10.899920 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-bundle\") pod \"60610957-4a9a-4a48-be3c-552281770c88\" (UID: \"60610957-4a9a-4a48-be3c-552281770c88\") " Jul 17 01:17:10.900348 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:10.900320 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-bundle" (OuterVolumeSpecName: "bundle") pod "60610957-4a9a-4a48-be3c-552281770c88" (UID: "60610957-4a9a-4a48-be3c-552281770c88"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:17:10.902039 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:10.902017 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60610957-4a9a-4a48-be3c-552281770c88-kube-api-access-9sh5d" (OuterVolumeSpecName: "kube-api-access-9sh5d") pod "60610957-4a9a-4a48-be3c-552281770c88" (UID: "60610957-4a9a-4a48-be3c-552281770c88"). InnerVolumeSpecName "kube-api-access-9sh5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:17:10.908111 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:10.908082 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-util" (OuterVolumeSpecName: "util") pod "60610957-4a9a-4a48-be3c-552281770c88" (UID: "60610957-4a9a-4a48-be3c-552281770c88"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:17:11.000460 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:11.000424 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-util\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:17:11.000460 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:11.000453 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sh5d\" (UniqueName: \"kubernetes.io/projected/60610957-4a9a-4a48-be3c-552281770c88-kube-api-access-9sh5d\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:17:11.000460 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:11.000464 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60610957-4a9a-4a48-be3c-552281770c88-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:17:11.089440 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:11.089345 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" event={"ID":"468703cc-6053-40d8-a5ab-262253de2ff8","Type":"ContainerStarted","Data":"784efa91412f753b4b14f78cdabe74b72e3a03a015326a400e1a254d7cf9504b"} Jul 17 01:17:11.091328 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:11.091301 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" event={"ID":"60610957-4a9a-4a48-be3c-552281770c88","Type":"ContainerDied","Data":"e4a2cce1389e760c2cd661ec4a03b707f72c28cc6c1572ffdb49f54be4c57c87"} Jul 17 01:17:11.091456 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:11.091344 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4a2cce1389e760c2cd661ec4a03b707f72c28cc6c1572ffdb49f54be4c57c87" Jul 17 01:17:11.091456 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:11.091349 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff82ww" Jul 17 01:17:11.105938 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:11.105895 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xtn7s" podStartSLOduration=1.483799382 podStartE2EDuration="4.105880244s" totalImagesPullingTime="2.622080862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:17:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:17:11.104006137 +0000 UTC m=+197.481930543" watchObservedRunningTime="2026-07-17 01:17:11.105880244 +0000 UTC m=+197.483804652" Jul 17 01:17:14.913983 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.913944 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8d5sb"] Jul 17 01:17:14.914591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.914367 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60610957-4a9a-4a48-be3c-552281770c88" containerName="util" Jul 17 01:17:14.914591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.914387 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60610957-4a9a-4a48-be3c-552281770c88" containerName="util" Jul 17 01:17:14.914591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.914413 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60610957-4a9a-4a48-be3c-552281770c88" containerName="pull" Jul 17 01:17:14.914591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.914422 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60610957-4a9a-4a48-be3c-552281770c88" containerName="pull" Jul 17 01:17:14.914591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.914441 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60610957-4a9a-4a48-be3c-552281770c88" containerName="extract" Jul 17 01:17:14.914591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.914449 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60610957-4a9a-4a48-be3c-552281770c88" containerName="extract" Jul 17 01:17:14.914591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.914521 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="60610957-4a9a-4a48-be3c-552281770c88" containerName="extract" Jul 17 01:17:14.917521 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.917500 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8d5sb" Jul 17 01:17:14.920007 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.919987 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9zfsw\"" Jul 17 01:17:14.924972 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:14.924951 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8d5sb"] Jul 17 01:17:15.033807 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:15.033767 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a81bf58-72ea-4edf-ad45-b930e8136b0f-bound-sa-token\") pod \"cert-manager-55b4d954d5-8d5sb\" (UID: \"4a81bf58-72ea-4edf-ad45-b930e8136b0f\") " pod="cert-manager/cert-manager-55b4d954d5-8d5sb" Jul 17 01:17:15.033807 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:15.033807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql5b2\" (UniqueName: \"kubernetes.io/projected/4a81bf58-72ea-4edf-ad45-b930e8136b0f-kube-api-access-ql5b2\") pod \"cert-manager-55b4d954d5-8d5sb\" (UID: \"4a81bf58-72ea-4edf-ad45-b930e8136b0f\") " pod="cert-manager/cert-manager-55b4d954d5-8d5sb" Jul 17 01:17:15.134508 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:15.134477 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a81bf58-72ea-4edf-ad45-b930e8136b0f-bound-sa-token\") pod \"cert-manager-55b4d954d5-8d5sb\" (UID: \"4a81bf58-72ea-4edf-ad45-b930e8136b0f\") " pod="cert-manager/cert-manager-55b4d954d5-8d5sb" Jul 17 01:17:15.134508 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:15.134511 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql5b2\" (UniqueName: \"kubernetes.io/projected/4a81bf58-72ea-4edf-ad45-b930e8136b0f-kube-api-access-ql5b2\") pod \"cert-manager-55b4d954d5-8d5sb\" (UID: \"4a81bf58-72ea-4edf-ad45-b930e8136b0f\") " pod="cert-manager/cert-manager-55b4d954d5-8d5sb" Jul 17 01:17:15.143116 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:15.143085 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a81bf58-72ea-4edf-ad45-b930e8136b0f-bound-sa-token\") pod \"cert-manager-55b4d954d5-8d5sb\" (UID: \"4a81bf58-72ea-4edf-ad45-b930e8136b0f\") " pod="cert-manager/cert-manager-55b4d954d5-8d5sb" Jul 17 01:17:15.143260 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:15.143215 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql5b2\" (UniqueName: \"kubernetes.io/projected/4a81bf58-72ea-4edf-ad45-b930e8136b0f-kube-api-access-ql5b2\") pod \"cert-manager-55b4d954d5-8d5sb\" (UID: \"4a81bf58-72ea-4edf-ad45-b930e8136b0f\") " pod="cert-manager/cert-manager-55b4d954d5-8d5sb" Jul 17 01:17:15.227218 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:15.227111 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-8d5sb" Jul 17 01:17:15.355781 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:15.355757 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-8d5sb"] Jul 17 01:17:15.358325 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:17:15.358290 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a81bf58_72ea_4edf_ad45_b930e8136b0f.slice/crio-686c425a727471259e821b0c9d0775232e001b82f6fe438c60cfcc1d586a0969 WatchSource:0}: Error finding container 686c425a727471259e821b0c9d0775232e001b82f6fe438c60cfcc1d586a0969: Status 404 returned error can't find the container with id 686c425a727471259e821b0c9d0775232e001b82f6fe438c60cfcc1d586a0969 Jul 17 01:17:16.111849 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:16.111814 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8d5sb" event={"ID":"4a81bf58-72ea-4edf-ad45-b930e8136b0f","Type":"ContainerStarted","Data":"e66ddde6dbdc57f5428eda00684e5e0e94ca2f6ba01b73667cfad03be9f310f1"} Jul 17 01:17:16.111849 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:16.111850 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-8d5sb" event={"ID":"4a81bf58-72ea-4edf-ad45-b930e8136b0f","Type":"ContainerStarted","Data":"686c425a727471259e821b0c9d0775232e001b82f6fe438c60cfcc1d586a0969"} Jul 17 01:17:16.126807 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:16.126758 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-8d5sb" podStartSLOduration=2.12673907 podStartE2EDuration="2.12673907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:17:16.125873404 +0000 UTC m=+202.503797810" watchObservedRunningTime="2026-07-17 01:17:16.12673907 +0000 UTC m=+202.504663478" Jul 17 01:17:22.133295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.133259 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b"] Jul 17 01:17:22.137009 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.136984 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.138793 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.138763 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b"] Jul 17 01:17:22.139525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.139502 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 17 01:17:22.139710 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.139687 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 17 01:17:22.140732 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.140696 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sgd4t\"" Jul 17 01:17:22.295438 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.295400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.295619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.295457 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.295619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.295509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zncf5\" (UniqueName: \"kubernetes.io/projected/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-kube-api-access-zncf5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.396758 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.396676 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.396758 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.396734 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zncf5\" (UniqueName: \"kubernetes.io/projected/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-kube-api-access-zncf5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.396900 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.396777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.397111 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.397093 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.397150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.397120 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.405065 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.405038 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zncf5\" (UniqueName: \"kubernetes.io/projected/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-kube-api-access-zncf5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.447625 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.447581 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:22.581374 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:22.581347 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b"] Jul 17 01:17:22.583844 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:17:22.583812 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c7df33_1a06_4ad3_951c_fc0302cff8a2.slice/crio-e5c19b0f8e58dcaba93c44ef95d2753a8e4b16db0605b4d0fe49f10983371eb0 WatchSource:0}: Error finding container e5c19b0f8e58dcaba93c44ef95d2753a8e4b16db0605b4d0fe49f10983371eb0: Status 404 returned error can't find the container with id e5c19b0f8e58dcaba93c44ef95d2753a8e4b16db0605b4d0fe49f10983371eb0 Jul 17 01:17:23.140789 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:23.140756 2564 generic.go:356] "Generic (PLEG): container finished" podID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerID="4b1ac3a0ea6d81acb43fd0abadc06d5daed82ee913ad2479fdaa309b3da99845" exitCode=0 Jul 17 01:17:23.141251 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:23.140842 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" event={"ID":"e0c7df33-1a06-4ad3-951c-fc0302cff8a2","Type":"ContainerDied","Data":"4b1ac3a0ea6d81acb43fd0abadc06d5daed82ee913ad2479fdaa309b3da99845"} Jul 17 01:17:23.141251 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:23.140900 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" event={"ID":"e0c7df33-1a06-4ad3-951c-fc0302cff8a2","Type":"ContainerStarted","Data":"e5c19b0f8e58dcaba93c44ef95d2753a8e4b16db0605b4d0fe49f10983371eb0"} Jul 17 01:17:24.146619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:24.146539 2564 generic.go:356] "Generic (PLEG): container finished" podID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerID="3797884c747f1546cfd48c4cf76d79249519bf9442a3f76b15390bc361347e89" exitCode=0 Jul 17 01:17:24.146955 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:24.146630 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" event={"ID":"e0c7df33-1a06-4ad3-951c-fc0302cff8a2","Type":"ContainerDied","Data":"3797884c747f1546cfd48c4cf76d79249519bf9442a3f76b15390bc361347e89"} Jul 17 01:17:25.152742 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:25.152703 2564 generic.go:356] "Generic (PLEG): container finished" podID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerID="a13a748415afcdaebe7d2b12ce7d8458c3c3a28a18c54587099edf746ff60ca2" exitCode=0 Jul 17 01:17:25.153201 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:25.152786 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" event={"ID":"e0c7df33-1a06-4ad3-951c-fc0302cff8a2","Type":"ContainerDied","Data":"a13a748415afcdaebe7d2b12ce7d8458c3c3a28a18c54587099edf746ff60ca2"} Jul 17 01:17:26.281430 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.281409 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:26.432652 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.432559 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-bundle\") pod \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " Jul 17 01:17:26.432652 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.432616 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zncf5\" (UniqueName: \"kubernetes.io/projected/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-kube-api-access-zncf5\") pod \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " Jul 17 01:17:26.432884 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.432676 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-util\") pod \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\" (UID: \"e0c7df33-1a06-4ad3-951c-fc0302cff8a2\") " Jul 17 01:17:26.433388 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.433361 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-bundle" (OuterVolumeSpecName: "bundle") pod "e0c7df33-1a06-4ad3-951c-fc0302cff8a2" (UID: "e0c7df33-1a06-4ad3-951c-fc0302cff8a2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:17:26.434893 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.434869 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-kube-api-access-zncf5" (OuterVolumeSpecName: "kube-api-access-zncf5") pod "e0c7df33-1a06-4ad3-951c-fc0302cff8a2" (UID: "e0c7df33-1a06-4ad3-951c-fc0302cff8a2"). InnerVolumeSpecName "kube-api-access-zncf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:17:26.438142 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.438117 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-util" (OuterVolumeSpecName: "util") pod "e0c7df33-1a06-4ad3-951c-fc0302cff8a2" (UID: "e0c7df33-1a06-4ad3-951c-fc0302cff8a2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:17:26.534096 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.534057 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:17:26.534096 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.534087 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zncf5\" (UniqueName: \"kubernetes.io/projected/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-kube-api-access-zncf5\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:17:26.534096 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:26.534096 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c7df33-1a06-4ad3-951c-fc0302cff8a2-util\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:17:27.162502 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:27.162471 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" Jul 17 01:17:27.162502 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:27.162489 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpjx2b" event={"ID":"e0c7df33-1a06-4ad3-951c-fc0302cff8a2","Type":"ContainerDied","Data":"e5c19b0f8e58dcaba93c44ef95d2753a8e4b16db0605b4d0fe49f10983371eb0"} Jul 17 01:17:27.162717 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:27.162542 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5c19b0f8e58dcaba93c44ef95d2753a8e4b16db0605b4d0fe49f10983371eb0" Jul 17 01:17:38.583545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.583508 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7"] Jul 17 01:17:38.583964 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.583947 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerName="util" Jul 17 01:17:38.583964 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.583965 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerName="util" Jul 17 01:17:38.584105 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.583977 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerName="pull" Jul 17 01:17:38.584105 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.583982 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerName="pull" Jul 17 01:17:38.584105 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.583994 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerName="extract" Jul 17 01:17:38.584105 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.583999 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerName="extract" Jul 17 01:17:38.584105 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.584046 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0c7df33-1a06-4ad3-951c-fc0302cff8a2" containerName="extract" Jul 17 01:17:38.591220 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.591083 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.593862 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.593841 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 17 01:17:38.593979 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.593907 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 17 01:17:38.593979 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.593928 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 17 01:17:38.594178 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.594146 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-276j7\"" Jul 17 01:17:38.594344 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.594327 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 17 01:17:38.604086 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.603990 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7"] Jul 17 01:17:38.743096 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.743056 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.743096 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.743102 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-manifests\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.743371 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.743209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-webhook-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.743371 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.743257 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-tmp\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.743371 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.743333 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5v7k\" (UniqueName: \"kubernetes.io/projected/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-kube-api-access-n5v7k\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.843845 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.843746 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-webhook-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.843845 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.843789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-tmp\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.844084 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.843870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5v7k\" (UniqueName: \"kubernetes.io/projected/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-kube-api-access-n5v7k\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.844084 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.843896 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.844084 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.843926 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-manifests\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.844433 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.844407 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-manifests\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.846334 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.846302 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-tmp\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.846461 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.846442 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-webhook-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.846506 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.846489 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.853347 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.853315 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5v7k\" (UniqueName: \"kubernetes.io/projected/be392a22-c9fe-4f4d-9b92-cfaff6acec2a-kube-api-access-n5v7k\") pod \"opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7\" (UID: \"be392a22-c9fe-4f4d-9b92-cfaff6acec2a\") " pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:38.901724 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:38.901691 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:39.043198 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:39.043142 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7"] Jul 17 01:17:39.046700 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:17:39.046663 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe392a22_c9fe_4f4d_9b92_cfaff6acec2a.slice/crio-baf2ca8ecbca78e14d1ad4b97c0978e497bcc136fa7610dd4bf388b179f4eba6 WatchSource:0}: Error finding container baf2ca8ecbca78e14d1ad4b97c0978e497bcc136fa7610dd4bf388b179f4eba6: Status 404 returned error can't find the container with id baf2ca8ecbca78e14d1ad4b97c0978e497bcc136fa7610dd4bf388b179f4eba6 Jul 17 01:17:39.212262 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:39.212229 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" event={"ID":"be392a22-c9fe-4f4d-9b92-cfaff6acec2a","Type":"ContainerStarted","Data":"baf2ca8ecbca78e14d1ad4b97c0978e497bcc136fa7610dd4bf388b179f4eba6"} Jul 17 01:17:42.229413 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:42.229369 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" event={"ID":"be392a22-c9fe-4f4d-9b92-cfaff6acec2a","Type":"ContainerStarted","Data":"9034bfa2154fde1a106c032057e69dc9241d20f66feda8466a74c4014b524481"} Jul 17 01:17:44.240447 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:44.240411 2564 generic.go:356] "Generic (PLEG): container finished" podID="be392a22-c9fe-4f4d-9b92-cfaff6acec2a" containerID="9034bfa2154fde1a106c032057e69dc9241d20f66feda8466a74c4014b524481" exitCode=0 Jul 17 01:17:44.240845 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:44.240491 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" event={"ID":"be392a22-c9fe-4f4d-9b92-cfaff6acec2a","Type":"ContainerDied","Data":"9034bfa2154fde1a106c032057e69dc9241d20f66feda8466a74c4014b524481"} Jul 17 01:17:45.247039 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:45.247000 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" event={"ID":"be392a22-c9fe-4f4d-9b92-cfaff6acec2a","Type":"ContainerStarted","Data":"ba19d3561867ef25e45ce864379530b3c5369f3133ea4e8faee81e2bd5fb0368"} Jul 17 01:17:45.247531 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:45.247326 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:45.272779 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:45.272724 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" podStartSLOduration=2.786079501 podStartE2EDuration="7.272705784s" totalImagesPullingTime="3.351453464s" totalInitContainerRuntime="1.135172819s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:17:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:17:45.271251857 +0000 UTC m=+231.649176266" watchObservedRunningTime="2026-07-17 01:17:45.272705784 +0000 UTC m=+231.650630191" Jul 17 01:17:56.254177 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:56.254131 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7" Jul 17 01:17:58.512337 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.512305 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n"] Jul 17 01:17:58.521056 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.521032 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.523640 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.523612 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 17 01:17:58.523640 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.523629 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 17 01:17:58.524311 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.524298 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 17 01:17:58.524541 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.524523 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 17 01:17:58.525028 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.525012 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-r6nnr\"" Jul 17 01:17:58.533751 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.533728 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n"] Jul 17 01:17:58.607593 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.607555 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92a62262-b71d-419f-be34-a58d94b1c43f-tmp\") pod \"kube-auth-proxy-68f596cfdd-tpm2n\" (UID: \"92a62262-b71d-419f-be34-a58d94b1c43f\") " pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.607786 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.607619 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92a62262-b71d-419f-be34-a58d94b1c43f-tls-certs\") pod \"kube-auth-proxy-68f596cfdd-tpm2n\" (UID: \"92a62262-b71d-419f-be34-a58d94b1c43f\") " pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.607786 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.607700 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5d65\" (UniqueName: \"kubernetes.io/projected/92a62262-b71d-419f-be34-a58d94b1c43f-kube-api-access-v5d65\") pod \"kube-auth-proxy-68f596cfdd-tpm2n\" (UID: \"92a62262-b71d-419f-be34-a58d94b1c43f\") " pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.708525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.708486 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5d65\" (UniqueName: \"kubernetes.io/projected/92a62262-b71d-419f-be34-a58d94b1c43f-kube-api-access-v5d65\") pod \"kube-auth-proxy-68f596cfdd-tpm2n\" (UID: \"92a62262-b71d-419f-be34-a58d94b1c43f\") " pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.708525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.708529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92a62262-b71d-419f-be34-a58d94b1c43f-tmp\") pod \"kube-auth-proxy-68f596cfdd-tpm2n\" (UID: \"92a62262-b71d-419f-be34-a58d94b1c43f\") " pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.708787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.708661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92a62262-b71d-419f-be34-a58d94b1c43f-tls-certs\") pod \"kube-auth-proxy-68f596cfdd-tpm2n\" (UID: \"92a62262-b71d-419f-be34-a58d94b1c43f\") " pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.710874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.710852 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/92a62262-b71d-419f-be34-a58d94b1c43f-tmp\") pod \"kube-auth-proxy-68f596cfdd-tpm2n\" (UID: \"92a62262-b71d-419f-be34-a58d94b1c43f\") " pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.711248 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.711227 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/92a62262-b71d-419f-be34-a58d94b1c43f-tls-certs\") pod \"kube-auth-proxy-68f596cfdd-tpm2n\" (UID: \"92a62262-b71d-419f-be34-a58d94b1c43f\") " pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.716628 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.716605 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5d65\" (UniqueName: \"kubernetes.io/projected/92a62262-b71d-419f-be34-a58d94b1c43f-kube-api-access-v5d65\") pod \"kube-auth-proxy-68f596cfdd-tpm2n\" (UID: \"92a62262-b71d-419f-be34-a58d94b1c43f\") " pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.804223 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.804121 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5dvqd"] Jul 17 01:17:58.807406 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.807390 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:17:58.814791 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.814767 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9x6c6\"" Jul 17 01:17:58.814913 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.814847 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 17 01:17:58.817080 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.817055 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5dvqd"] Jul 17 01:17:58.829747 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.829717 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" Jul 17 01:17:58.910939 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.910904 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lr5z\" (UniqueName: \"kubernetes.io/projected/7454a501-b0b8-4509-9a4f-a71396becbb3-kube-api-access-8lr5z\") pod \"odh-model-controller-7998c67dc4-5dvqd\" (UID: \"7454a501-b0b8-4509-9a4f-a71396becbb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:17:58.911086 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.911022 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7454a501-b0b8-4509-9a4f-a71396becbb3-cert\") pod \"odh-model-controller-7998c67dc4-5dvqd\" (UID: \"7454a501-b0b8-4509-9a4f-a71396becbb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:17:58.971686 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:58.971649 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n"] Jul 17 01:17:58.974945 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:17:58.974870 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92a62262_b71d_419f_be34_a58d94b1c43f.slice/crio-f9afcee9f6fe5baf6c16d8cc7056324d6438c0622f91652a07991cea063ca50a WatchSource:0}: Error finding container f9afcee9f6fe5baf6c16d8cc7056324d6438c0622f91652a07991cea063ca50a: Status 404 returned error can't find the container with id f9afcee9f6fe5baf6c16d8cc7056324d6438c0622f91652a07991cea063ca50a Jul 17 01:17:59.011986 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.011952 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lr5z\" (UniqueName: \"kubernetes.io/projected/7454a501-b0b8-4509-9a4f-a71396becbb3-kube-api-access-8lr5z\") pod \"odh-model-controller-7998c67dc4-5dvqd\" (UID: \"7454a501-b0b8-4509-9a4f-a71396becbb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:17:59.012203 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.012050 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7454a501-b0b8-4509-9a4f-a71396becbb3-cert\") pod \"odh-model-controller-7998c67dc4-5dvqd\" (UID: \"7454a501-b0b8-4509-9a4f-a71396becbb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:17:59.012267 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:17:59.012219 2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 17 01:17:59.012315 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:17:59.012305 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7454a501-b0b8-4509-9a4f-a71396becbb3-cert podName:7454a501-b0b8-4509-9a4f-a71396becbb3 nodeName:}" failed. No retries permitted until 2026-07-17 01:17:59.51228338 +0000 UTC m=+245.890207766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7454a501-b0b8-4509-9a4f-a71396becbb3-cert") pod "odh-model-controller-7998c67dc4-5dvqd" (UID: "7454a501-b0b8-4509-9a4f-a71396becbb3") : secret "odh-model-controller-webhook-cert" not found Jul 17 01:17:59.020945 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.020923 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lr5z\" (UniqueName: \"kubernetes.io/projected/7454a501-b0b8-4509-9a4f-a71396becbb3-kube-api-access-8lr5z\") pod \"odh-model-controller-7998c67dc4-5dvqd\" (UID: \"7454a501-b0b8-4509-9a4f-a71396becbb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:17:59.305776 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.305739 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" event={"ID":"92a62262-b71d-419f-be34-a58d94b1c43f","Type":"ContainerStarted","Data":"f9afcee9f6fe5baf6c16d8cc7056324d6438c0622f91652a07991cea063ca50a"} Jul 17 01:17:59.516696 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.516659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7454a501-b0b8-4509-9a4f-a71396becbb3-cert\") pod \"odh-model-controller-7998c67dc4-5dvqd\" (UID: \"7454a501-b0b8-4509-9a4f-a71396becbb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:17:59.519244 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.519216 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7454a501-b0b8-4509-9a4f-a71396becbb3-cert\") pod \"odh-model-controller-7998c67dc4-5dvqd\" (UID: \"7454a501-b0b8-4509-9a4f-a71396becbb3\") " pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:17:59.662877 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.662837 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-65z88"] Jul 17 01:17:59.665818 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.665796 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:17:59.668869 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.668844 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-cpkc7\"" Jul 17 01:17:59.668999 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.668913 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 17 01:17:59.680198 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.680147 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-65z88"] Jul 17 01:17:59.718612 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.718574 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:17:59.718950 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.718926 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faa1066e-2d83-46f2-aefe-ed11fad61da0-tls-certs\") pod \"model-serving-api-54d89bd5fc-65z88\" (UID: \"faa1066e-2d83-46f2-aefe-ed11fad61da0\") " pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:17:59.719062 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.718983 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pch2f\" (UniqueName: \"kubernetes.io/projected/faa1066e-2d83-46f2-aefe-ed11fad61da0-kube-api-access-pch2f\") pod \"model-serving-api-54d89bd5fc-65z88\" (UID: \"faa1066e-2d83-46f2-aefe-ed11fad61da0\") " pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:17:59.750408 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.750375 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9"] Jul 17 01:17:59.755017 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.754995 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.758271 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.758229 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 17 01:17:59.758271 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.758258 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 17 01:17:59.758598 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.758563 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vgcn8\"" Jul 17 01:17:59.758714 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.758624 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 17 01:17:59.758917 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.758765 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 17 01:17:59.774736 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.774694 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9"] Jul 17 01:17:59.819693 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.819661 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-token\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.819874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.819714 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faa1066e-2d83-46f2-aefe-ed11fad61da0-tls-certs\") pod \"model-serving-api-54d89bd5fc-65z88\" (UID: \"faa1066e-2d83-46f2-aefe-ed11fad61da0\") " pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:17:59.819874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.819756 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pch2f\" (UniqueName: \"kubernetes.io/projected/faa1066e-2d83-46f2-aefe-ed11fad61da0-kube-api-access-pch2f\") pod \"model-serving-api-54d89bd5fc-65z88\" (UID: \"faa1066e-2d83-46f2-aefe-ed11fad61da0\") " pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:17:59.819874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.819785 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82f6878-fa57-4b9e-894a-64a822b6f20c-local-certs\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.819874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.819809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.819874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.819856 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.820129 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:17:59.819875 2564 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 17 01:17:59.820129 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.819879 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv9bk\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-kube-api-access-gv9bk\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.820129 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:17:59.819948 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa1066e-2d83-46f2-aefe-ed11fad61da0-tls-certs podName:faa1066e-2d83-46f2-aefe-ed11fad61da0 nodeName:}" failed. No retries permitted until 2026-07-17 01:18:00.319926005 +0000 UTC m=+246.697850399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/faa1066e-2d83-46f2-aefe-ed11fad61da0-tls-certs") pod "model-serving-api-54d89bd5fc-65z88" (UID: "faa1066e-2d83-46f2-aefe-ed11fad61da0") : secret "model-serving-api-tls" not found Jul 17 01:17:59.820129 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.820013 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-cacerts\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.820129 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.820097 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.837363 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.837332 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pch2f\" (UniqueName: \"kubernetes.io/projected/faa1066e-2d83-46f2-aefe-ed11fad61da0-kube-api-access-pch2f\") pod \"model-serving-api-54d89bd5fc-65z88\" (UID: \"faa1066e-2d83-46f2-aefe-ed11fad61da0\") " pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:17:59.865708 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.865657 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-5dvqd"] Jul 17 01:17:59.867819 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:17:59.867793 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7454a501_b0b8_4509_9a4f_a71396becbb3.slice/crio-c9382a51fd9859210af684024c9d84d1cdb0f7834b5228a02b2f3941236596db WatchSource:0}: Error finding container c9382a51fd9859210af684024c9d84d1cdb0f7834b5228a02b2f3941236596db: Status 404 returned error can't find the container with id c9382a51fd9859210af684024c9d84d1cdb0f7834b5228a02b2f3941236596db Jul 17 01:17:59.921049 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.920943 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.921049 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.920995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv9bk\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-kube-api-access-gv9bk\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.921049 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.921031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-cacerts\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.921355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.921117 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.921355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.921151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-token\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.921355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.921251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82f6878-fa57-4b9e-894a-64a822b6f20c-local-certs\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.921355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.921274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.922007 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.921944 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.923962 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.923921 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.924085 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.924045 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82f6878-fa57-4b9e-894a-64a822b6f20c-local-certs\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.924186 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.924128 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-cacerts\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.924232 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.924195 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.930270 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.930246 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv9bk\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-kube-api-access-gv9bk\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:17:59.930431 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:17:59.930409 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-token\") pod \"istiod-openshift-gateway-7574459656-cl6x9\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:18:00.067326 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:00.067285 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:18:00.226461 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:00.226378 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9"] Jul 17 01:18:00.229423 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:18:00.229396 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82f6878_fa57_4b9e_894a_64a822b6f20c.slice/crio-5ed8756d015401ce5a11499456cfeb81753352fa7ecece2db470f2df099c0c7f WatchSource:0}: Error finding container 5ed8756d015401ce5a11499456cfeb81753352fa7ecece2db470f2df099c0c7f: Status 404 returned error can't find the container with id 5ed8756d015401ce5a11499456cfeb81753352fa7ecece2db470f2df099c0c7f Jul 17 01:18:00.311539 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:00.311499 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" event={"ID":"c82f6878-fa57-4b9e-894a-64a822b6f20c","Type":"ContainerStarted","Data":"5ed8756d015401ce5a11499456cfeb81753352fa7ecece2db470f2df099c0c7f"} Jul 17 01:18:00.312704 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:00.312676 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" event={"ID":"7454a501-b0b8-4509-9a4f-a71396becbb3","Type":"ContainerStarted","Data":"c9382a51fd9859210af684024c9d84d1cdb0f7834b5228a02b2f3941236596db"} Jul 17 01:18:00.324109 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:00.324077 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faa1066e-2d83-46f2-aefe-ed11fad61da0-tls-certs\") pod \"model-serving-api-54d89bd5fc-65z88\" (UID: \"faa1066e-2d83-46f2-aefe-ed11fad61da0\") " pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:18:00.326538 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:00.326513 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/faa1066e-2d83-46f2-aefe-ed11fad61da0-tls-certs\") pod \"model-serving-api-54d89bd5fc-65z88\" (UID: \"faa1066e-2d83-46f2-aefe-ed11fad61da0\") " pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:18:00.578307 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:00.578215 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:18:00.734491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:00.733491 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-65z88"] Jul 17 01:18:00.739769 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:18:00.739732 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa1066e_2d83_46f2_aefe_ed11fad61da0.slice/crio-6a55d99492576f9be1eb30a2d1a0bc491962a8533f41b5c9772ec46a192004ee WatchSource:0}: Error finding container 6a55d99492576f9be1eb30a2d1a0bc491962a8533f41b5c9772ec46a192004ee: Status 404 returned error can't find the container with id 6a55d99492576f9be1eb30a2d1a0bc491962a8533f41b5c9772ec46a192004ee Jul 17 01:18:01.323692 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:01.323611 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-65z88" event={"ID":"faa1066e-2d83-46f2-aefe-ed11fad61da0","Type":"ContainerStarted","Data":"6a55d99492576f9be1eb30a2d1a0bc491962a8533f41b5c9772ec46a192004ee"} Jul 17 01:18:04.885738 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:04.885693 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:18:04.886057 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:04.885782 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:18:05.347949 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.347907 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" event={"ID":"92a62262-b71d-419f-be34-a58d94b1c43f","Type":"ContainerStarted","Data":"14fe885490d94da67d53761d54959272ceae699cbb10aedbee437d1db96d1259"} Jul 17 01:18:05.349689 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.349653 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" event={"ID":"c82f6878-fa57-4b9e-894a-64a822b6f20c","Type":"ContainerStarted","Data":"1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f"} Jul 17 01:18:05.349863 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.349847 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:18:05.351150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.351124 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" event={"ID":"7454a501-b0b8-4509-9a4f-a71396becbb3","Type":"ContainerStarted","Data":"b066d3f2c133475182d17399ace6f87e0eb7ed9828384d234dd238bd7f17dd11"} Jul 17 01:18:05.351279 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.351200 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:18:05.351892 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.351869 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-cl6x9 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 01:18:05.351983 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.351920 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" podUID="c82f6878-fa57-4b9e-894a-64a822b6f20c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 01:18:05.352814 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.352791 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-65z88" event={"ID":"faa1066e-2d83-46f2-aefe-ed11fad61da0","Type":"ContainerStarted","Data":"103e9a852b6d7acecdec15b1fe917ecaa659c7a0d7e21fedd1adea27a289d104"} Jul 17 01:18:05.352932 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.352915 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:18:05.377956 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.377911 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-68f596cfdd-tpm2n" podStartSLOduration=1.486015716 podStartE2EDuration="7.377895907s" totalImagesPullingTime="5.891880191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:18:05.376829548 +0000 UTC m=+251.754753966" watchObservedRunningTime="2026-07-17 01:18:05.377895907 +0000 UTC m=+251.755820323" Jul 17 01:18:05.398909 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.398864 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" podStartSLOduration=1.74469897 podStartE2EDuration="6.398846945s" totalImagesPullingTime="4.654147975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:18:05.398147402 +0000 UTC m=+251.776071811" watchObservedRunningTime="2026-07-17 01:18:05.398846945 +0000 UTC m=+251.776771354" Jul 17 01:18:05.417724 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.417650 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" podStartSLOduration=2.410260361 podStartE2EDuration="7.417630116s" totalImagesPullingTime="5.007369755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:18:05.414808178 +0000 UTC m=+251.792732587" watchObservedRunningTime="2026-07-17 01:18:05.417630116 +0000 UTC m=+251.795554526" Jul 17 01:18:05.450683 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:05.450617 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-65z88" podStartSLOduration=2.316950761 podStartE2EDuration="6.4505941s" totalImagesPullingTime="4.133643339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:17:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:18:05.448036166 +0000 UTC m=+251.825960574" watchObservedRunningTime="2026-07-17 01:18:05.4505941 +0000 UTC m=+251.828518511" Jul 17 01:18:06.358594 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:06.358565 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:18:10.459398 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.459361 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9"] Jul 17 01:18:10.459784 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.459547 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" podUID="c82f6878-fa57-4b9e-894a-64a822b6f20c" containerName="discovery" containerID="cri-o://1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f" gracePeriod=30 Jul 17 01:18:10.696231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.696207 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:18:10.821045 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.820949 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-kubeconfig\") pod \"c82f6878-fa57-4b9e-894a-64a822b6f20c\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " Jul 17 01:18:10.821045 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.821003 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-cacerts\") pod \"c82f6878-fa57-4b9e-894a-64a822b6f20c\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " Jul 17 01:18:10.821045 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.821035 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82f6878-fa57-4b9e-894a-64a822b6f20c-local-certs\") pod \"c82f6878-fa57-4b9e-894a-64a822b6f20c\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " Jul 17 01:18:10.821332 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.821089 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gv9bk\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-kube-api-access-gv9bk\") pod \"c82f6878-fa57-4b9e-894a-64a822b6f20c\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " Jul 17 01:18:10.821332 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.821113 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-dns-cert\") pod \"c82f6878-fa57-4b9e-894a-64a822b6f20c\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " Jul 17 01:18:10.821332 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.821156 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-ca-configmap\") pod \"c82f6878-fa57-4b9e-894a-64a822b6f20c\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " Jul 17 01:18:10.821332 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.821228 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-token\") pod \"c82f6878-fa57-4b9e-894a-64a822b6f20c\" (UID: \"c82f6878-fa57-4b9e-894a-64a822b6f20c\") " Jul 17 01:18:10.821865 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.821659 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "c82f6878-fa57-4b9e-894a-64a822b6f20c" (UID: "c82f6878-fa57-4b9e-894a-64a822b6f20c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:18:10.823489 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.823464 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "c82f6878-fa57-4b9e-894a-64a822b6f20c" (UID: "c82f6878-fa57-4b9e-894a-64a822b6f20c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:18:10.823619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.823566 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-token" (OuterVolumeSpecName: "istio-token") pod "c82f6878-fa57-4b9e-894a-64a822b6f20c" (UID: "c82f6878-fa57-4b9e-894a-64a822b6f20c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:18:10.823689 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.823669 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-cacerts" (OuterVolumeSpecName: "cacerts") pod "c82f6878-fa57-4b9e-894a-64a822b6f20c" (UID: "c82f6878-fa57-4b9e-894a-64a822b6f20c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:18:10.823731 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.823680 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c82f6878-fa57-4b9e-894a-64a822b6f20c-local-certs" (OuterVolumeSpecName: "local-certs") pod "c82f6878-fa57-4b9e-894a-64a822b6f20c" (UID: "c82f6878-fa57-4b9e-894a-64a822b6f20c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:18:10.823886 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.823859 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-kube-api-access-gv9bk" (OuterVolumeSpecName: "kube-api-access-gv9bk") pod "c82f6878-fa57-4b9e-894a-64a822b6f20c" (UID: "c82f6878-fa57-4b9e-894a-64a822b6f20c"). InnerVolumeSpecName "kube-api-access-gv9bk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:18:10.823886 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.823860 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "c82f6878-fa57-4b9e-894a-64a822b6f20c" (UID: "c82f6878-fa57-4b9e-894a-64a822b6f20c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:18:10.922422 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.922375 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-ca-configmap\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:18:10.922422 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.922415 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:18:10.922422 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.922425 2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-kubeconfig\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:18:10.922422 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.922434 2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-cacerts\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:18:10.922422 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.922441 2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82f6878-fa57-4b9e-894a-64a822b6f20c-local-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:18:10.922717 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.922449 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv9bk\" (UniqueName: \"kubernetes.io/projected/c82f6878-fa57-4b9e-894a-64a822b6f20c-kube-api-access-gv9bk\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:18:10.922717 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:10.922457 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c82f6878-fa57-4b9e-894a-64a822b6f20c-istio-csr-dns-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:18:11.382700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:11.382669 2564 generic.go:356] "Generic (PLEG): container finished" podID="c82f6878-fa57-4b9e-894a-64a822b6f20c" containerID="1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f" exitCode=0 Jul 17 01:18:11.382865 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:11.382717 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" Jul 17 01:18:11.382865 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:11.382743 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" event={"ID":"c82f6878-fa57-4b9e-894a-64a822b6f20c","Type":"ContainerDied","Data":"1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f"} Jul 17 01:18:11.382865 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:11.382782 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9" event={"ID":"c82f6878-fa57-4b9e-894a-64a822b6f20c","Type":"ContainerDied","Data":"5ed8756d015401ce5a11499456cfeb81753352fa7ecece2db470f2df099c0c7f"} Jul 17 01:18:11.382865 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:11.382802 2564 scope.go:117] "RemoveContainer" containerID="1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f" Jul 17 01:18:11.391827 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:11.391809 2564 scope.go:117] "RemoveContainer" containerID="1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f" Jul 17 01:18:11.392065 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:18:11.392048 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f\": container with ID starting with 1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f not found: ID does not exist" containerID="1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f" Jul 17 01:18:11.392111 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:11.392072 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f"} err="failed to get container status \"1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f\": rpc error: code = NotFound desc = could not find container \"1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f\": container with ID starting with 1d2b252d89f4f202f3d78bf0c7fd8147b38ffec6ba759cf6afccae604fd0fc2f not found: ID does not exist" Jul 17 01:18:11.403952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:11.403927 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9"] Jul 17 01:18:11.412700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:11.412678 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-cl6x9"] Jul 17 01:18:12.133177 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:12.133134 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c82f6878-fa57-4b9e-894a-64a822b6f20c" path="/var/lib/kubelet/pods/c82f6878-fa57-4b9e-894a-64a822b6f20c/volumes" Jul 17 01:18:16.360333 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:16.360259 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-5dvqd" Jul 17 01:18:16.362137 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:16.362118 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-65z88" Jul 17 01:18:53.992823 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:53.992791 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:18:54.002772 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:54.002746 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 17 01:18:54.006721 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:54.006704 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:18:59.556645 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.556602 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmjcn"] Jul 17 01:18:59.558035 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.557085 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c82f6878-fa57-4b9e-894a-64a822b6f20c" containerName="discovery" Jul 17 01:18:59.558035 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.557099 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c82f6878-fa57-4b9e-894a-64a822b6f20c" containerName="discovery" Jul 17 01:18:59.558035 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.557180 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c82f6878-fa57-4b9e-894a-64a822b6f20c" containerName="discovery" Jul 17 01:18:59.559152 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.559132 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" Jul 17 01:18:59.561749 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.561730 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 01:18:59.562924 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.562890 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 01:18:59.563025 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.562921 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-97vrz\"" Jul 17 01:18:59.569361 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.569339 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmjcn"] Jul 17 01:18:59.641973 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.641930 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52dzm\" (UniqueName: \"kubernetes.io/projected/13667db1-33ce-4861-97c8-4a0b7dc7d182-kube-api-access-52dzm\") pod \"kuadrant-operator-catalog-jmjcn\" (UID: \"13667db1-33ce-4861-97c8-4a0b7dc7d182\") " pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" Jul 17 01:18:59.743372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.743335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52dzm\" (UniqueName: \"kubernetes.io/projected/13667db1-33ce-4861-97c8-4a0b7dc7d182-kube-api-access-52dzm\") pod \"kuadrant-operator-catalog-jmjcn\" (UID: \"13667db1-33ce-4861-97c8-4a0b7dc7d182\") " pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" Jul 17 01:18:59.751569 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.751535 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52dzm\" (UniqueName: \"kubernetes.io/projected/13667db1-33ce-4861-97c8-4a0b7dc7d182-kube-api-access-52dzm\") pod \"kuadrant-operator-catalog-jmjcn\" (UID: \"13667db1-33ce-4861-97c8-4a0b7dc7d182\") " pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" Jul 17 01:18:59.870813 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.870730 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" Jul 17 01:18:59.927128 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:18:59.927100 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmjcn"] Jul 17 01:19:00.002540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.001560 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmjcn"] Jul 17 01:19:00.004545 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:00.004511 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13667db1_33ce_4861_97c8_4a0b7dc7d182.slice/crio-b1f2a777ca5fbe5ecefc7315209ed3d5bc240979901812c004a92782f52b131d WatchSource:0}: Error finding container b1f2a777ca5fbe5ecefc7315209ed3d5bc240979901812c004a92782f52b131d: Status 404 returned error can't find the container with id b1f2a777ca5fbe5ecefc7315209ed3d5bc240979901812c004a92782f52b131d Jul 17 01:19:00.005967 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.005949 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 01:19:00.133529 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.133453 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hsq69"] Jul 17 01:19:00.136092 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.136070 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hsq69" Jul 17 01:19:00.139881 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.139843 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hsq69"] Jul 17 01:19:00.246906 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.246863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jn5k\" (UniqueName: \"kubernetes.io/projected/40010bf0-a051-4574-9a4d-3bd215921dce-kube-api-access-9jn5k\") pod \"kuadrant-operator-catalog-hsq69\" (UID: \"40010bf0-a051-4574-9a4d-3bd215921dce\") " pod="kuadrant-system/kuadrant-operator-catalog-hsq69" Jul 17 01:19:00.347887 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.347840 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jn5k\" (UniqueName: \"kubernetes.io/projected/40010bf0-a051-4574-9a4d-3bd215921dce-kube-api-access-9jn5k\") pod \"kuadrant-operator-catalog-hsq69\" (UID: \"40010bf0-a051-4574-9a4d-3bd215921dce\") " pod="kuadrant-system/kuadrant-operator-catalog-hsq69" Jul 17 01:19:00.357067 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.357042 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jn5k\" (UniqueName: \"kubernetes.io/projected/40010bf0-a051-4574-9a4d-3bd215921dce-kube-api-access-9jn5k\") pod \"kuadrant-operator-catalog-hsq69\" (UID: \"40010bf0-a051-4574-9a4d-3bd215921dce\") " pod="kuadrant-system/kuadrant-operator-catalog-hsq69" Jul 17 01:19:00.447066 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.447032 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hsq69" Jul 17 01:19:00.581602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.581575 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hsq69"] Jul 17 01:19:00.583747 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:00.583722 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40010bf0_a051_4574_9a4d_3bd215921dce.slice/crio-96621e75cf5d6557ce2b79852eb799a440b0ba1471c3cbe09968a13547b4728d WatchSource:0}: Error finding container 96621e75cf5d6557ce2b79852eb799a440b0ba1471c3cbe09968a13547b4728d: Status 404 returned error can't find the container with id 96621e75cf5d6557ce2b79852eb799a440b0ba1471c3cbe09968a13547b4728d Jul 17 01:19:00.607570 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.607517 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hsq69" event={"ID":"40010bf0-a051-4574-9a4d-3bd215921dce","Type":"ContainerStarted","Data":"96621e75cf5d6557ce2b79852eb799a440b0ba1471c3cbe09968a13547b4728d"} Jul 17 01:19:00.608673 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:00.608649 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" event={"ID":"13667db1-33ce-4861-97c8-4a0b7dc7d182","Type":"ContainerStarted","Data":"b1f2a777ca5fbe5ecefc7315209ed3d5bc240979901812c004a92782f52b131d"} Jul 17 01:19:02.619557 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:02.619447 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" event={"ID":"13667db1-33ce-4861-97c8-4a0b7dc7d182","Type":"ContainerStarted","Data":"a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8"} Jul 17 01:19:02.619557 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:02.619470 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" podUID="13667db1-33ce-4861-97c8-4a0b7dc7d182" containerName="registry-server" containerID="cri-o://a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8" gracePeriod=2 Jul 17 01:19:02.621150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:02.621121 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hsq69" event={"ID":"40010bf0-a051-4574-9a4d-3bd215921dce","Type":"ContainerStarted","Data":"38be565952f2cf4f0e19753431853c3e268f9346bf920ffadb9409bef59a793a"} Jul 17 01:19:02.634075 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:02.634023 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" podStartSLOduration=1.3596114369999999 podStartE2EDuration="3.634004906s" totalImagesPullingTime="2.274393469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:19:02.633625569 +0000 UTC m=+309.011549977" watchObservedRunningTime="2026-07-17 01:19:02.634004906 +0000 UTC m=+309.011929314" Jul 17 01:19:02.649743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:02.649690 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hsq69" podStartSLOduration=0.951342333 podStartE2EDuration="2.649674716s" totalImagesPullingTime="1.698332383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:19:02.648285599 +0000 UTC m=+309.026210030" watchObservedRunningTime="2026-07-17 01:19:02.649674716 +0000 UTC m=+309.027599124" Jul 17 01:19:02.861409 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:02.861383 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" Jul 17 01:19:02.973245 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:02.973211 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52dzm\" (UniqueName: \"kubernetes.io/projected/13667db1-33ce-4861-97c8-4a0b7dc7d182-kube-api-access-52dzm\") pod \"13667db1-33ce-4861-97c8-4a0b7dc7d182\" (UID: \"13667db1-33ce-4861-97c8-4a0b7dc7d182\") " Jul 17 01:19:02.975458 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:02.975428 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13667db1-33ce-4861-97c8-4a0b7dc7d182-kube-api-access-52dzm" (OuterVolumeSpecName: "kube-api-access-52dzm") pod "13667db1-33ce-4861-97c8-4a0b7dc7d182" (UID: "13667db1-33ce-4861-97c8-4a0b7dc7d182"). InnerVolumeSpecName "kube-api-access-52dzm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:19:03.074010 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.073974 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52dzm\" (UniqueName: \"kubernetes.io/projected/13667db1-33ce-4861-97c8-4a0b7dc7d182-kube-api-access-52dzm\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:03.627197 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.627094 2564 generic.go:356] "Generic (PLEG): container finished" podID="13667db1-33ce-4861-97c8-4a0b7dc7d182" containerID="a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8" exitCode=0 Jul 17 01:19:03.627197 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.627139 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" Jul 17 01:19:03.627633 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.627198 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" event={"ID":"13667db1-33ce-4861-97c8-4a0b7dc7d182","Type":"ContainerDied","Data":"a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8"} Jul 17 01:19:03.627633 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.627247 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jmjcn" event={"ID":"13667db1-33ce-4861-97c8-4a0b7dc7d182","Type":"ContainerDied","Data":"b1f2a777ca5fbe5ecefc7315209ed3d5bc240979901812c004a92782f52b131d"} Jul 17 01:19:03.627633 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.627268 2564 scope.go:117] "RemoveContainer" containerID="a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8" Jul 17 01:19:03.636206 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.636188 2564 scope.go:117] "RemoveContainer" containerID="a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8" Jul 17 01:19:03.636511 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:19:03.636491 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8\": container with ID starting with a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8 not found: ID does not exist" containerID="a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8" Jul 17 01:19:03.636553 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.636520 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8"} err="failed to get container status \"a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8\": rpc error: code = NotFound desc = could not find container \"a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8\": container with ID starting with a2cea1b020c6478da4fd0707415757816dd7cc47798770dbde66966dfc3ce6f8 not found: ID does not exist" Jul 17 01:19:03.646670 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.646645 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmjcn"] Jul 17 01:19:03.648971 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:03.648951 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmjcn"] Jul 17 01:19:04.132879 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:04.132843 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13667db1-33ce-4861-97c8-4a0b7dc7d182" path="/var/lib/kubelet/pods/13667db1-33ce-4861-97c8-4a0b7dc7d182/volumes" Jul 17 01:19:10.447811 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:10.447771 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hsq69" Jul 17 01:19:10.448286 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:10.447857 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hsq69" Jul 17 01:19:10.470434 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:10.470409 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hsq69" Jul 17 01:19:10.681938 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:10.681908 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hsq69" Jul 17 01:19:14.961490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:14.961451 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9"] Jul 17 01:19:14.961869 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:14.961771 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13667db1-33ce-4861-97c8-4a0b7dc7d182" containerName="registry-server" Jul 17 01:19:14.961869 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:14.961782 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="13667db1-33ce-4861-97c8-4a0b7dc7d182" containerName="registry-server" Jul 17 01:19:14.961869 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:14.961830 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="13667db1-33ce-4861-97c8-4a0b7dc7d182" containerName="registry-server" Jul 17 01:19:14.965012 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:14.964995 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:14.967544 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:14.967520 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wmsvn\"" Jul 17 01:19:14.972345 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:14.972012 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9"] Jul 17 01:19:15.078885 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.078845 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.078885 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.078886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.079088 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.078924 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkw52\" (UniqueName: \"kubernetes.io/projected/9b8599f9-59db-4d51-8072-d9ad0265d377-kube-api-access-vkw52\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.180273 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.180239 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.180273 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.180274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.180471 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.180315 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkw52\" (UniqueName: \"kubernetes.io/projected/9b8599f9-59db-4d51-8072-d9ad0265d377-kube-api-access-vkw52\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.180723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.180700 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.180759 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.180716 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.188692 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.188669 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkw52\" (UniqueName: \"kubernetes.io/projected/9b8599f9-59db-4d51-8072-d9ad0265d377-kube-api-access-vkw52\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.275719 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.275636 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:15.405127 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.405102 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9"] Jul 17 01:19:15.407071 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:15.407048 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8599f9_59db_4d51_8072_d9ad0265d377.slice/crio-c298f52e8d31fdf08d762cfdc21efb87b49ea146272e683ee2687c18f03be29c WatchSource:0}: Error finding container c298f52e8d31fdf08d762cfdc21efb87b49ea146272e683ee2687c18f03be29c: Status 404 returned error can't find the container with id c298f52e8d31fdf08d762cfdc21efb87b49ea146272e683ee2687c18f03be29c Jul 17 01:19:15.563352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.563274 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj"] Jul 17 01:19:15.566563 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.566548 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.573913 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.573888 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj"] Jul 17 01:19:15.683503 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.683468 2564 generic.go:356] "Generic (PLEG): container finished" podID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerID="4f5ad31365706e1147eb2abe8b2b797e608e4145a1f8a667590f0248be9c728f" exitCode=0 Jul 17 01:19:15.683702 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.683523 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" event={"ID":"9b8599f9-59db-4d51-8072-d9ad0265d377","Type":"ContainerDied","Data":"4f5ad31365706e1147eb2abe8b2b797e608e4145a1f8a667590f0248be9c728f"} Jul 17 01:19:15.683702 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.683567 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" event={"ID":"9b8599f9-59db-4d51-8072-d9ad0265d377","Type":"ContainerStarted","Data":"c298f52e8d31fdf08d762cfdc21efb87b49ea146272e683ee2687c18f03be29c"} Jul 17 01:19:15.684498 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.684441 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.684553 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.684506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.684553 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.684543 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzz79\" (UniqueName: \"kubernetes.io/projected/4715a21c-afee-42d1-9be5-a389f32fb399-kube-api-access-wzz79\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.785373 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.785295 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.785589 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.785463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.785589 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.785536 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzz79\" (UniqueName: \"kubernetes.io/projected/4715a21c-afee-42d1-9be5-a389f32fb399-kube-api-access-wzz79\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.785801 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.785781 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.785874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.785854 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.794071 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.794051 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzz79\" (UniqueName: \"kubernetes.io/projected/4715a21c-afee-42d1-9be5-a389f32fb399-kube-api-access-wzz79\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.876783 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.876702 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:15.958776 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.958740 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7"] Jul 17 01:19:15.963605 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.963577 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:15.970591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:15.970562 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7"] Jul 17 01:19:16.015276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.015247 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj"] Jul 17 01:19:16.016698 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:16.016674 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4715a21c_afee_42d1_9be5_a389f32fb399.slice/crio-fc31c8da88d0da02543a9593f356ded57bdeec08034fabc0072613af2e58e0cd WatchSource:0}: Error finding container fc31c8da88d0da02543a9593f356ded57bdeec08034fabc0072613af2e58e0cd: Status 404 returned error can't find the container with id fc31c8da88d0da02543a9593f356ded57bdeec08034fabc0072613af2e58e0cd Jul 17 01:19:16.088645 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.088618 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrrrk\" (UniqueName: \"kubernetes.io/projected/bc530462-42b3-47f6-ba41-ed4ff81c17de-kube-api-access-zrrrk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.088766 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.088690 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.088816 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.088759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.189832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.189801 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrrrk\" (UniqueName: \"kubernetes.io/projected/bc530462-42b3-47f6-ba41-ed4ff81c17de-kube-api-access-zrrrk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.189999 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.189870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.189999 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.189970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.190336 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.190317 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.190380 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.190332 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.199009 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.198987 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrrrk\" (UniqueName: \"kubernetes.io/projected/bc530462-42b3-47f6-ba41-ed4ff81c17de-kube-api-access-zrrrk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.275785 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.275755 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:16.366332 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.366274 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj"] Jul 17 01:19:16.370759 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.370713 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.376915 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.376828 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj"] Jul 17 01:19:16.418917 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.418896 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7"] Jul 17 01:19:16.420246 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:16.420219 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc530462_42b3_47f6_ba41_ed4ff81c17de.slice/crio-dbd283b5aafc0d18d752d41068ffb7016717c89bbe2b52cf943f78c6e539d880 WatchSource:0}: Error finding container dbd283b5aafc0d18d752d41068ffb7016717c89bbe2b52cf943f78c6e539d880: Status 404 returned error can't find the container with id dbd283b5aafc0d18d752d41068ffb7016717c89bbe2b52cf943f78c6e539d880 Jul 17 01:19:16.493288 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.493265 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.493391 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.493328 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.493457 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.493433 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpwgq\" (UniqueName: \"kubernetes.io/projected/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-kube-api-access-tpwgq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.594051 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.594023 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.594191 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.594073 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.594191 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.594113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpwgq\" (UniqueName: \"kubernetes.io/projected/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-kube-api-access-tpwgq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.594408 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.594390 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.594480 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.594461 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.602448 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.602428 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpwgq\" (UniqueName: \"kubernetes.io/projected/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-kube-api-access-tpwgq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.684478 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.684448 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:16.691685 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.691656 2564 generic.go:356] "Generic (PLEG): container finished" podID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerID="29ee17488cdf9393b86386c6af6fbc4ea03a01971031afaeff8602acc67dedce" exitCode=0 Jul 17 01:19:16.691827 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.691744 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" event={"ID":"9b8599f9-59db-4d51-8072-d9ad0265d377","Type":"ContainerDied","Data":"29ee17488cdf9393b86386c6af6fbc4ea03a01971031afaeff8602acc67dedce"} Jul 17 01:19:16.693232 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.693211 2564 generic.go:356] "Generic (PLEG): container finished" podID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerID="6bc09407a0641bf5b1ca3058592b696355a347378d46efcd52c95a9922bbfabd" exitCode=0 Jul 17 01:19:16.693337 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.693274 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" event={"ID":"bc530462-42b3-47f6-ba41-ed4ff81c17de","Type":"ContainerDied","Data":"6bc09407a0641bf5b1ca3058592b696355a347378d46efcd52c95a9922bbfabd"} Jul 17 01:19:16.693337 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.693307 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" event={"ID":"bc530462-42b3-47f6-ba41-ed4ff81c17de","Type":"ContainerStarted","Data":"dbd283b5aafc0d18d752d41068ffb7016717c89bbe2b52cf943f78c6e539d880"} Jul 17 01:19:16.695024 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.695003 2564 generic.go:356] "Generic (PLEG): container finished" podID="4715a21c-afee-42d1-9be5-a389f32fb399" containerID="649e1c2d6aa42c5da0d4ac232787041ac4851e1693cd20c867e283e11cfeb815" exitCode=0 Jul 17 01:19:16.695138 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.695113 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" event={"ID":"4715a21c-afee-42d1-9be5-a389f32fb399","Type":"ContainerDied","Data":"649e1c2d6aa42c5da0d4ac232787041ac4851e1693cd20c867e283e11cfeb815"} Jul 17 01:19:16.695252 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.695189 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" event={"ID":"4715a21c-afee-42d1-9be5-a389f32fb399","Type":"ContainerStarted","Data":"fc31c8da88d0da02543a9593f356ded57bdeec08034fabc0072613af2e58e0cd"} Jul 17 01:19:16.824736 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:16.824709 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj"] Jul 17 01:19:16.876203 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:16.876158 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02fd5e6c_69f1_47bb_ae16_b848053e2dfa.slice/crio-7057ecbf08910bdf8e9ec8d5dedf697db148624303a9ef895c0c0604e44234ea WatchSource:0}: Error finding container 7057ecbf08910bdf8e9ec8d5dedf697db148624303a9ef895c0c0604e44234ea: Status 404 returned error can't find the container with id 7057ecbf08910bdf8e9ec8d5dedf697db148624303a9ef895c0c0604e44234ea Jul 17 01:19:17.702178 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:17.702132 2564 generic.go:356] "Generic (PLEG): container finished" podID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerID="9544f43266305f8e7620ccc5f567437f40f335f46dab42512b193ba41e63715d" exitCode=0 Jul 17 01:19:17.702574 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:17.702217 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" event={"ID":"9b8599f9-59db-4d51-8072-d9ad0265d377","Type":"ContainerDied","Data":"9544f43266305f8e7620ccc5f567437f40f335f46dab42512b193ba41e63715d"} Jul 17 01:19:17.704012 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:17.703985 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" event={"ID":"bc530462-42b3-47f6-ba41-ed4ff81c17de","Type":"ContainerStarted","Data":"ad7472b194d5186179e066f7893f801e9388730509be70ce584f7c7f141070fc"} Jul 17 01:19:17.705774 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:17.705751 2564 generic.go:356] "Generic (PLEG): container finished" podID="4715a21c-afee-42d1-9be5-a389f32fb399" containerID="b00cfb49f7afaecf394fd75bf90b59df0eaeca6b9001b1d3085a75eadd98e554" exitCode=0 Jul 17 01:19:17.705862 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:17.705804 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" event={"ID":"4715a21c-afee-42d1-9be5-a389f32fb399","Type":"ContainerDied","Data":"b00cfb49f7afaecf394fd75bf90b59df0eaeca6b9001b1d3085a75eadd98e554"} Jul 17 01:19:17.714283 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:17.714256 2564 generic.go:356] "Generic (PLEG): container finished" podID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerID="055ab1609353c60c9440c13f51b1f4b009e615098025bc6d4d958b6d74c9f207" exitCode=0 Jul 17 01:19:17.714388 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:17.714353 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" event={"ID":"02fd5e6c-69f1-47bb-ae16-b848053e2dfa","Type":"ContainerDied","Data":"055ab1609353c60c9440c13f51b1f4b009e615098025bc6d4d958b6d74c9f207"} Jul 17 01:19:17.714444 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:17.714391 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" event={"ID":"02fd5e6c-69f1-47bb-ae16-b848053e2dfa","Type":"ContainerStarted","Data":"7057ecbf08910bdf8e9ec8d5dedf697db148624303a9ef895c0c0604e44234ea"} Jul 17 01:19:18.721736 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:18.721705 2564 generic.go:356] "Generic (PLEG): container finished" podID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerID="ad7472b194d5186179e066f7893f801e9388730509be70ce584f7c7f141070fc" exitCode=0 Jul 17 01:19:18.722143 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:18.721776 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" event={"ID":"bc530462-42b3-47f6-ba41-ed4ff81c17de","Type":"ContainerDied","Data":"ad7472b194d5186179e066f7893f801e9388730509be70ce584f7c7f141070fc"} Jul 17 01:19:18.723893 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:18.723867 2564 generic.go:356] "Generic (PLEG): container finished" podID="4715a21c-afee-42d1-9be5-a389f32fb399" containerID="6bcadeeaec3e9f89aca7184b1b7108e18bd2559ae0f18412de75d7380ee02b64" exitCode=0 Jul 17 01:19:18.724050 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:18.723950 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" event={"ID":"4715a21c-afee-42d1-9be5-a389f32fb399","Type":"ContainerDied","Data":"6bcadeeaec3e9f89aca7184b1b7108e18bd2559ae0f18412de75d7380ee02b64"} Jul 17 01:19:18.725582 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:18.725553 2564 generic.go:356] "Generic (PLEG): container finished" podID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerID="7bb22f97a2877d26a18517b23e249b12f6bbb234a71ab8e2ee27c9aa6c8dd0d7" exitCode=0 Jul 17 01:19:18.725686 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:18.725614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" event={"ID":"02fd5e6c-69f1-47bb-ae16-b848053e2dfa","Type":"ContainerDied","Data":"7bb22f97a2877d26a18517b23e249b12f6bbb234a71ab8e2ee27c9aa6c8dd0d7"} Jul 17 01:19:18.862597 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:18.861924 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:19.013537 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.013504 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-bundle\") pod \"9b8599f9-59db-4d51-8072-d9ad0265d377\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " Jul 17 01:19:19.013709 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.013556 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkw52\" (UniqueName: \"kubernetes.io/projected/9b8599f9-59db-4d51-8072-d9ad0265d377-kube-api-access-vkw52\") pod \"9b8599f9-59db-4d51-8072-d9ad0265d377\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " Jul 17 01:19:19.013709 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.013578 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-util\") pod \"9b8599f9-59db-4d51-8072-d9ad0265d377\" (UID: \"9b8599f9-59db-4d51-8072-d9ad0265d377\") " Jul 17 01:19:19.014065 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.014026 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-bundle" (OuterVolumeSpecName: "bundle") pod "9b8599f9-59db-4d51-8072-d9ad0265d377" (UID: "9b8599f9-59db-4d51-8072-d9ad0265d377"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:19:19.015780 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.015756 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8599f9-59db-4d51-8072-d9ad0265d377-kube-api-access-vkw52" (OuterVolumeSpecName: "kube-api-access-vkw52") pod "9b8599f9-59db-4d51-8072-d9ad0265d377" (UID: "9b8599f9-59db-4d51-8072-d9ad0265d377"). InnerVolumeSpecName "kube-api-access-vkw52". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:19:19.019004 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.018970 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-util" (OuterVolumeSpecName: "util") pod "9b8599f9-59db-4d51-8072-d9ad0265d377" (UID: "9b8599f9-59db-4d51-8072-d9ad0265d377"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:19:19.115056 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.114964 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:19.115056 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.114996 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkw52\" (UniqueName: \"kubernetes.io/projected/9b8599f9-59db-4d51-8072-d9ad0265d377-kube-api-access-vkw52\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:19.115056 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.115007 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9b8599f9-59db-4d51-8072-d9ad0265d377-util\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:19.732932 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.732899 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" event={"ID":"9b8599f9-59db-4d51-8072-d9ad0265d377","Type":"ContainerDied","Data":"c298f52e8d31fdf08d762cfdc21efb87b49ea146272e683ee2687c18f03be29c"} Jul 17 01:19:19.733413 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.732963 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c298f52e8d31fdf08d762cfdc21efb87b49ea146272e683ee2687c18f03be29c" Jul 17 01:19:19.733413 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.732909 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9" Jul 17 01:19:19.734873 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.734850 2564 generic.go:356] "Generic (PLEG): container finished" podID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerID="3203afe364aae23d17c9afe8d73895e9e0f78f624f4ccfb7ae6b21eae1b3fe7d" exitCode=0 Jul 17 01:19:19.735005 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.734927 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" event={"ID":"bc530462-42b3-47f6-ba41-ed4ff81c17de","Type":"ContainerDied","Data":"3203afe364aae23d17c9afe8d73895e9e0f78f624f4ccfb7ae6b21eae1b3fe7d"} Jul 17 01:19:19.736798 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.736775 2564 generic.go:356] "Generic (PLEG): container finished" podID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerID="e22f469cd085e7ee6f9e73e4cd5fc2065a903289d26fe032c7b1437bd23537c9" exitCode=0 Jul 17 01:19:19.736905 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.736857 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" event={"ID":"02fd5e6c-69f1-47bb-ae16-b848053e2dfa","Type":"ContainerDied","Data":"e22f469cd085e7ee6f9e73e4cd5fc2065a903289d26fe032c7b1437bd23537c9"} Jul 17 01:19:19.869425 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:19.869400 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:20.024761 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.024674 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-bundle\") pod \"4715a21c-afee-42d1-9be5-a389f32fb399\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " Jul 17 01:19:20.024761 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.024726 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-util\") pod \"4715a21c-afee-42d1-9be5-a389f32fb399\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " Jul 17 01:19:20.024936 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.024822 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wzz79\" (UniqueName: \"kubernetes.io/projected/4715a21c-afee-42d1-9be5-a389f32fb399-kube-api-access-wzz79\") pod \"4715a21c-afee-42d1-9be5-a389f32fb399\" (UID: \"4715a21c-afee-42d1-9be5-a389f32fb399\") " Jul 17 01:19:20.025301 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.025274 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-bundle" (OuterVolumeSpecName: "bundle") pod "4715a21c-afee-42d1-9be5-a389f32fb399" (UID: "4715a21c-afee-42d1-9be5-a389f32fb399"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:19:20.026946 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.026914 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4715a21c-afee-42d1-9be5-a389f32fb399-kube-api-access-wzz79" (OuterVolumeSpecName: "kube-api-access-wzz79") pod "4715a21c-afee-42d1-9be5-a389f32fb399" (UID: "4715a21c-afee-42d1-9be5-a389f32fb399"). InnerVolumeSpecName "kube-api-access-wzz79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:19:20.029754 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.029719 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-util" (OuterVolumeSpecName: "util") pod "4715a21c-afee-42d1-9be5-a389f32fb399" (UID: "4715a21c-afee-42d1-9be5-a389f32fb399"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:19:20.125475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.125434 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzz79\" (UniqueName: \"kubernetes.io/projected/4715a21c-afee-42d1-9be5-a389f32fb399-kube-api-access-wzz79\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:20.125475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.125470 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:20.125475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.125481 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4715a21c-afee-42d1-9be5-a389f32fb399-util\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:20.742998 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.742969 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" Jul 17 01:19:20.743487 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.743001 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj" event={"ID":"4715a21c-afee-42d1-9be5-a389f32fb399","Type":"ContainerDied","Data":"fc31c8da88d0da02543a9593f356ded57bdeec08034fabc0072613af2e58e0cd"} Jul 17 01:19:20.743487 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.743046 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc31c8da88d0da02543a9593f356ded57bdeec08034fabc0072613af2e58e0cd" Jul 17 01:19:20.890703 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.890679 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:20.892621 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:20.892602 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:21.032526 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.032440 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpwgq\" (UniqueName: \"kubernetes.io/projected/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-kube-api-access-tpwgq\") pod \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " Jul 17 01:19:21.032526 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.032481 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-bundle\") pod \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " Jul 17 01:19:21.032696 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.032537 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-util\") pod \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\" (UID: \"02fd5e6c-69f1-47bb-ae16-b848053e2dfa\") " Jul 17 01:19:21.032696 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.032643 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-util\") pod \"bc530462-42b3-47f6-ba41-ed4ff81c17de\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " Jul 17 01:19:21.032770 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.032720 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-bundle\") pod \"bc530462-42b3-47f6-ba41-ed4ff81c17de\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " Jul 17 01:19:21.032770 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.032759 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrrrk\" (UniqueName: \"kubernetes.io/projected/bc530462-42b3-47f6-ba41-ed4ff81c17de-kube-api-access-zrrrk\") pod \"bc530462-42b3-47f6-ba41-ed4ff81c17de\" (UID: \"bc530462-42b3-47f6-ba41-ed4ff81c17de\") " Jul 17 01:19:21.033205 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.033090 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-bundle" (OuterVolumeSpecName: "bundle") pod "02fd5e6c-69f1-47bb-ae16-b848053e2dfa" (UID: "02fd5e6c-69f1-47bb-ae16-b848053e2dfa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:19:21.033480 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.033312 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-bundle" (OuterVolumeSpecName: "bundle") pod "bc530462-42b3-47f6-ba41-ed4ff81c17de" (UID: "bc530462-42b3-47f6-ba41-ed4ff81c17de"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:19:21.034822 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.034801 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-kube-api-access-tpwgq" (OuterVolumeSpecName: "kube-api-access-tpwgq") pod "02fd5e6c-69f1-47bb-ae16-b848053e2dfa" (UID: "02fd5e6c-69f1-47bb-ae16-b848053e2dfa"). InnerVolumeSpecName "kube-api-access-tpwgq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:19:21.034899 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.034850 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc530462-42b3-47f6-ba41-ed4ff81c17de-kube-api-access-zrrrk" (OuterVolumeSpecName: "kube-api-access-zrrrk") pod "bc530462-42b3-47f6-ba41-ed4ff81c17de" (UID: "bc530462-42b3-47f6-ba41-ed4ff81c17de"). InnerVolumeSpecName "kube-api-access-zrrrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:19:21.037640 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.037621 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-util" (OuterVolumeSpecName: "util") pod "02fd5e6c-69f1-47bb-ae16-b848053e2dfa" (UID: "02fd5e6c-69f1-47bb-ae16-b848053e2dfa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:19:21.041082 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.041053 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-util" (OuterVolumeSpecName: "util") pod "bc530462-42b3-47f6-ba41-ed4ff81c17de" (UID: "bc530462-42b3-47f6-ba41-ed4ff81c17de"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:19:21.133379 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.133345 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:21.133379 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.133373 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrrrk\" (UniqueName: \"kubernetes.io/projected/bc530462-42b3-47f6-ba41-ed4ff81c17de-kube-api-access-zrrrk\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:21.133379 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.133383 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpwgq\" (UniqueName: \"kubernetes.io/projected/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-kube-api-access-tpwgq\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:21.133591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.133393 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:21.133591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.133400 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/02fd5e6c-69f1-47bb-ae16-b848053e2dfa-util\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:21.133591 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.133407 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc530462-42b3-47f6-ba41-ed4ff81c17de-util\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:19:21.749480 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.749456 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" Jul 17 01:19:21.749899 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.749457 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7" event={"ID":"bc530462-42b3-47f6-ba41-ed4ff81c17de","Type":"ContainerDied","Data":"dbd283b5aafc0d18d752d41068ffb7016717c89bbe2b52cf943f78c6e539d880"} Jul 17 01:19:21.749899 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.749602 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbd283b5aafc0d18d752d41068ffb7016717c89bbe2b52cf943f78c6e539d880" Jul 17 01:19:21.751400 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.751385 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" Jul 17 01:19:21.751494 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.751384 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj" event={"ID":"02fd5e6c-69f1-47bb-ae16-b848053e2dfa","Type":"ContainerDied","Data":"7057ecbf08910bdf8e9ec8d5dedf697db148624303a9ef895c0c0604e44234ea"} Jul 17 01:19:21.751534 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:21.751493 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7057ecbf08910bdf8e9ec8d5dedf697db148624303a9ef895c0c0604e44234ea" Jul 17 01:19:30.950242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950205 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj"] Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950510 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4715a21c-afee-42d1-9be5-a389f32fb399" containerName="util" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950521 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4715a21c-afee-42d1-9be5-a389f32fb399" containerName="util" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950536 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950541 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950549 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerName="pull" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950554 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerName="pull" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950559 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4715a21c-afee-42d1-9be5-a389f32fb399" containerName="pull" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950564 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4715a21c-afee-42d1-9be5-a389f32fb399" containerName="pull" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950569 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerName="pull" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950573 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerName="pull" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950579 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerName="util" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950584 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerName="util" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950589 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerName="util" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950594 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerName="util" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950599 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4715a21c-afee-42d1-9be5-a389f32fb399" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950603 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4715a21c-afee-42d1-9be5-a389f32fb399" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950610 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950616 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950621 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerName="util" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950626 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerName="util" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950632 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950636 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950647 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerName="pull" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950653 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerName="pull" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950717 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc530462-42b3-47f6-ba41-ed4ff81c17de" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950724 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b8599f9-59db-4d51-8072-d9ad0265d377" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950731 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4715a21c-afee-42d1-9be5-a389f32fb399" containerName="extract" Jul 17 01:19:30.950723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.950737 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="02fd5e6c-69f1-47bb-ae16-b848053e2dfa" containerName="extract" Jul 17 01:19:30.955105 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.955089 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" Jul 17 01:19:30.957965 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.957941 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7rjt8\"" Jul 17 01:19:30.958099 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.957977 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 17 01:19:30.963503 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:30.963480 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj"] Jul 17 01:19:31.008815 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:31.008773 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chm6j\" (UniqueName: \"kubernetes.io/projected/01b979c3-1417-403e-8a15-6fd43b91e033-kube-api-access-chm6j\") pod \"dns-operator-controller-manager-864dd78c4c-bn4fj\" (UID: \"01b979c3-1417-403e-8a15-6fd43b91e033\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" Jul 17 01:19:31.109231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:31.109191 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chm6j\" (UniqueName: \"kubernetes.io/projected/01b979c3-1417-403e-8a15-6fd43b91e033-kube-api-access-chm6j\") pod \"dns-operator-controller-manager-864dd78c4c-bn4fj\" (UID: \"01b979c3-1417-403e-8a15-6fd43b91e033\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" Jul 17 01:19:31.120232 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:31.120197 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chm6j\" (UniqueName: \"kubernetes.io/projected/01b979c3-1417-403e-8a15-6fd43b91e033-kube-api-access-chm6j\") pod \"dns-operator-controller-manager-864dd78c4c-bn4fj\" (UID: \"01b979c3-1417-403e-8a15-6fd43b91e033\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" Jul 17 01:19:31.266430 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:31.266341 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" Jul 17 01:19:31.404252 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:31.404228 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj"] Jul 17 01:19:31.406581 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:31.406541 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01b979c3_1417_403e_8a15_6fd43b91e033.slice/crio-3fece7c473badc22a4db5e2bc1c82326f142b30ad9dc31b418deacb30bb32233 WatchSource:0}: Error finding container 3fece7c473badc22a4db5e2bc1c82326f142b30ad9dc31b418deacb30bb32233: Status 404 returned error can't find the container with id 3fece7c473badc22a4db5e2bc1c82326f142b30ad9dc31b418deacb30bb32233 Jul 17 01:19:31.801130 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:31.801096 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" event={"ID":"01b979c3-1417-403e-8a15-6fd43b91e033","Type":"ContainerStarted","Data":"3fece7c473badc22a4db5e2bc1c82326f142b30ad9dc31b418deacb30bb32233"} Jul 17 01:19:34.335699 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.335665 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd"] Jul 17 01:19:34.338917 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.338899 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:19:34.341394 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.341375 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-b94sl\"" Jul 17 01:19:34.348300 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.348275 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd"] Jul 17 01:19:34.438854 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.438819 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2sw8\" (UniqueName: \"kubernetes.io/projected/8ca3dbca-df84-4262-b37b-536f4abf5e5d-kube-api-access-n2sw8\") pod \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" (UID: \"8ca3dbca-df84-4262-b37b-536f4abf5e5d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:19:34.540303 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.540258 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2sw8\" (UniqueName: \"kubernetes.io/projected/8ca3dbca-df84-4262-b37b-536f4abf5e5d-kube-api-access-n2sw8\") pod \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" (UID: \"8ca3dbca-df84-4262-b37b-536f4abf5e5d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:19:34.548836 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.548811 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2sw8\" (UniqueName: \"kubernetes.io/projected/8ca3dbca-df84-4262-b37b-536f4abf5e5d-kube-api-access-n2sw8\") pod \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" (UID: \"8ca3dbca-df84-4262-b37b-536f4abf5e5d\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:19:34.649940 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.649849 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:19:34.787502 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.787478 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd"] Jul 17 01:19:34.789115 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:34.789087 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca3dbca_df84_4262_b37b_536f4abf5e5d.slice/crio-aeb8d3a5e22c14a9c41bff2a8c2e4e3e79101396bf4d7f5a97493beac5863545 WatchSource:0}: Error finding container aeb8d3a5e22c14a9c41bff2a8c2e4e3e79101396bf4d7f5a97493beac5863545: Status 404 returned error can't find the container with id aeb8d3a5e22c14a9c41bff2a8c2e4e3e79101396bf4d7f5a97493beac5863545 Jul 17 01:19:34.819606 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.819572 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" event={"ID":"8ca3dbca-df84-4262-b37b-536f4abf5e5d","Type":"ContainerStarted","Data":"aeb8d3a5e22c14a9c41bff2a8c2e4e3e79101396bf4d7f5a97493beac5863545"} Jul 17 01:19:34.821147 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.821116 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" event={"ID":"01b979c3-1417-403e-8a15-6fd43b91e033","Type":"ContainerStarted","Data":"88ed2984d28143e7760b844dfda2d04a6d5c8d0831036640f6bb71bdc3a04066"} Jul 17 01:19:34.821265 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.821238 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" Jul 17 01:19:34.839223 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:34.839151 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" podStartSLOduration=2.385614565 podStartE2EDuration="4.839133219s" totalImagesPullingTime="2.453518654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:19:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:19:34.836999628 +0000 UTC m=+341.214924037" watchObservedRunningTime="2026-07-17 01:19:34.839133219 +0000 UTC m=+341.217057623" Jul 17 01:19:36.832960 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:36.832909 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" event={"ID":"8ca3dbca-df84-4262-b37b-536f4abf5e5d","Type":"ContainerStarted","Data":"6f1633daa973ee09f550e8bfb146d51592db85383080b451a7b784e2163bcfbf"} Jul 17 01:19:36.833437 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:36.833009 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:19:36.850443 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:36.850391 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" podStartSLOduration=1.329334175 podStartE2EDuration="2.850376874s" totalImagesPullingTime="1.521042699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:19:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:19:36.849383347 +0000 UTC m=+343.227307755" watchObservedRunningTime="2026-07-17 01:19:36.850376874 +0000 UTC m=+343.228301282" Jul 17 01:19:39.034452 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.034415 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27"] Jul 17 01:19:39.037870 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.037852 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:39.040503 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.040480 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2x5dk\"" Jul 17 01:19:39.048517 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.048487 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27"] Jul 17 01:19:39.078099 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.078059 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlqlb\" (UniqueName: \"kubernetes.io/projected/db15149f-4284-4d16-a7f6-39b40037856f-kube-api-access-mlqlb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" (UID: \"db15149f-4284-4d16-a7f6-39b40037856f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:39.078300 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.078107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db15149f-4284-4d16-a7f6-39b40037856f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" (UID: \"db15149f-4284-4d16-a7f6-39b40037856f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:39.179577 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.179539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlqlb\" (UniqueName: \"kubernetes.io/projected/db15149f-4284-4d16-a7f6-39b40037856f-kube-api-access-mlqlb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" (UID: \"db15149f-4284-4d16-a7f6-39b40037856f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:39.179763 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.179598 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db15149f-4284-4d16-a7f6-39b40037856f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" (UID: \"db15149f-4284-4d16-a7f6-39b40037856f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:39.180097 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.180067 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db15149f-4284-4d16-a7f6-39b40037856f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" (UID: \"db15149f-4284-4d16-a7f6-39b40037856f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:39.187456 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.187430 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlqlb\" (UniqueName: \"kubernetes.io/projected/db15149f-4284-4d16-a7f6-39b40037856f-kube-api-access-mlqlb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" (UID: \"db15149f-4284-4d16-a7f6-39b40037856f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:39.349107 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.349012 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:39.486132 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.486109 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27"] Jul 17 01:19:39.488452 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:39.488424 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb15149f_4284_4d16_a7f6_39b40037856f.slice/crio-5d6b4abb4c8c8c58bb2846f1f95ae98362152bb48e28457f78776c8cbfd8f4e7 WatchSource:0}: Error finding container 5d6b4abb4c8c8c58bb2846f1f95ae98362152bb48e28457f78776c8cbfd8f4e7: Status 404 returned error can't find the container with id 5d6b4abb4c8c8c58bb2846f1f95ae98362152bb48e28457f78776c8cbfd8f4e7 Jul 17 01:19:39.847992 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:39.847957 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" event={"ID":"db15149f-4284-4d16-a7f6-39b40037856f","Type":"ContainerStarted","Data":"5d6b4abb4c8c8c58bb2846f1f95ae98362152bb48e28457f78776c8cbfd8f4e7"} Jul 17 01:19:42.457668 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:42.457630 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4vktj"] Jul 17 01:19:42.461002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:42.460985 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" Jul 17 01:19:42.464247 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:42.464226 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-n7df4\"" Jul 17 01:19:42.474595 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:42.474567 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4vktj"] Jul 17 01:19:42.506999 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:42.506958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t4sq\" (UniqueName: \"kubernetes.io/projected/83788c1d-38da-49f2-bd19-6b48eceb4cc0-kube-api-access-7t4sq\") pod \"authorino-operator-75cc8d5cb7-4vktj\" (UID: \"83788c1d-38da-49f2-bd19-6b48eceb4cc0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" Jul 17 01:19:42.607537 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:42.607498 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t4sq\" (UniqueName: \"kubernetes.io/projected/83788c1d-38da-49f2-bd19-6b48eceb4cc0-kube-api-access-7t4sq\") pod \"authorino-operator-75cc8d5cb7-4vktj\" (UID: \"83788c1d-38da-49f2-bd19-6b48eceb4cc0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" Jul 17 01:19:42.619432 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:42.619396 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t4sq\" (UniqueName: \"kubernetes.io/projected/83788c1d-38da-49f2-bd19-6b48eceb4cc0-kube-api-access-7t4sq\") pod \"authorino-operator-75cc8d5cb7-4vktj\" (UID: \"83788c1d-38da-49f2-bd19-6b48eceb4cc0\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" Jul 17 01:19:42.771474 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:42.771373 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" Jul 17 01:19:42.940094 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:42.940069 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4vktj"] Jul 17 01:19:42.941979 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:42.941939 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83788c1d_38da_49f2_bd19_6b48eceb4cc0.slice/crio-6befccc7c60a3ca92dae8efae375fb6e2b78a921b05e61cc1c63961135d79611 WatchSource:0}: Error finding container 6befccc7c60a3ca92dae8efae375fb6e2b78a921b05e61cc1c63961135d79611: Status 404 returned error can't find the container with id 6befccc7c60a3ca92dae8efae375fb6e2b78a921b05e61cc1c63961135d79611 Jul 17 01:19:43.873446 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:43.873372 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" event={"ID":"83788c1d-38da-49f2-bd19-6b48eceb4cc0","Type":"ContainerStarted","Data":"6befccc7c60a3ca92dae8efae375fb6e2b78a921b05e61cc1c63961135d79611"} Jul 17 01:19:45.828999 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:45.828963 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bn4fj" Jul 17 01:19:46.892068 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:46.891968 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" event={"ID":"db15149f-4284-4d16-a7f6-39b40037856f","Type":"ContainerStarted","Data":"35f6ca3d2bfd978a8cb1919bd4a2979481d8ca656562e6eac71c3acd8ac51846"} Jul 17 01:19:46.892538 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:46.892153 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:46.893526 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:46.893501 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" event={"ID":"83788c1d-38da-49f2-bd19-6b48eceb4cc0","Type":"ContainerStarted","Data":"7288be7045eaea2f102dad9fdebcfb311dc3ebfc2f1e20cdd11ad4c549f6140f"} Jul 17 01:19:46.893629 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:46.893618 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" Jul 17 01:19:46.912137 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:46.912085 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" podStartSLOduration=0.933615652 podStartE2EDuration="7.912069649s" totalImagesPullingTime="6.978453997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:19:46.910117666 +0000 UTC m=+353.288042075" watchObservedRunningTime="2026-07-17 01:19:46.912069649 +0000 UTC m=+353.289994057" Jul 17 01:19:46.926692 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:46.926640 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" podStartSLOduration=1.412387593 podStartE2EDuration="4.926622986s" totalImagesPullingTime="3.514235393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:19:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:19:46.925700641 +0000 UTC m=+353.303625049" watchObservedRunningTime="2026-07-17 01:19:46.926622986 +0000 UTC m=+353.304547394" Jul 17 01:19:47.839865 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:47.839835 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:19:49.246848 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.246815 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6"] Jul 17 01:19:49.250704 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.250680 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.253092 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.253066 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 17 01:19:49.253246 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.253066 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-wmsvn\"" Jul 17 01:19:49.253323 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.253075 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 17 01:19:49.260810 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.260784 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6"] Jul 17 01:19:49.369396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.369353 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f55e9a4-88bb-45fe-99c3-7e073d28f7c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g7cx6\" (UID: \"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.369396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.369399 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8f55e9a4-88bb-45fe-99c3-7e073d28f7c1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-g7cx6\" (UID: \"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.369655 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.369419 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fzwm\" (UniqueName: \"kubernetes.io/projected/8f55e9a4-88bb-45fe-99c3-7e073d28f7c1-kube-api-access-6fzwm\") pod \"kuadrant-console-plugin-66b5774985-g7cx6\" (UID: \"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.470394 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.470354 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f55e9a4-88bb-45fe-99c3-7e073d28f7c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g7cx6\" (UID: \"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.470394 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.470398 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8f55e9a4-88bb-45fe-99c3-7e073d28f7c1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-g7cx6\" (UID: \"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.470648 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.470419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fzwm\" (UniqueName: \"kubernetes.io/projected/8f55e9a4-88bb-45fe-99c3-7e073d28f7c1-kube-api-access-6fzwm\") pod \"kuadrant-console-plugin-66b5774985-g7cx6\" (UID: \"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.471002 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.470981 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8f55e9a4-88bb-45fe-99c3-7e073d28f7c1-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-g7cx6\" (UID: \"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.473013 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.472990 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8f55e9a4-88bb-45fe-99c3-7e073d28f7c1-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-g7cx6\" (UID: \"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.478738 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.478715 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fzwm\" (UniqueName: \"kubernetes.io/projected/8f55e9a4-88bb-45fe-99c3-7e073d28f7c1-kube-api-access-6fzwm\") pod \"kuadrant-console-plugin-66b5774985-g7cx6\" (UID: \"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.561503 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.561413 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" Jul 17 01:19:49.694063 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.694038 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6"] Jul 17 01:19:49.695790 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:19:49.695752 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f55e9a4_88bb_45fe_99c3_7e073d28f7c1.slice/crio-3c98ca1ba5a72ee05dddfc4de2d35ba768a3bffcc45469cd586e65f36ed1037a WatchSource:0}: Error finding container 3c98ca1ba5a72ee05dddfc4de2d35ba768a3bffcc45469cd586e65f36ed1037a: Status 404 returned error can't find the container with id 3c98ca1ba5a72ee05dddfc4de2d35ba768a3bffcc45469cd586e65f36ed1037a Jul 17 01:19:49.910121 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:49.910089 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" event={"ID":"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1","Type":"ContainerStarted","Data":"3c98ca1ba5a72ee05dddfc4de2d35ba768a3bffcc45469cd586e65f36ed1037a"} Jul 17 01:19:57.900969 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:57.900927 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:57.901448 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:57.901015 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4vktj" Jul 17 01:19:58.952036 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:58.952003 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w"] Jul 17 01:19:58.965619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:58.965595 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:19:58.989187 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:58.974205 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w"] Jul 17 01:19:59.059371 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.059317 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hmz4\" (UniqueName: \"kubernetes.io/projected/58f7c632-2d25-4cb1-803a-23d73503f6c9-kube-api-access-9hmz4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" (UID: \"58f7c632-2d25-4cb1-803a-23d73503f6c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:19:59.059552 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.059391 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58f7c632-2d25-4cb1-803a-23d73503f6c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" (UID: \"58f7c632-2d25-4cb1-803a-23d73503f6c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:19:59.160407 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.160375 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58f7c632-2d25-4cb1-803a-23d73503f6c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" (UID: \"58f7c632-2d25-4cb1-803a-23d73503f6c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:19:59.160572 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.160450 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hmz4\" (UniqueName: \"kubernetes.io/projected/58f7c632-2d25-4cb1-803a-23d73503f6c9-kube-api-access-9hmz4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" (UID: \"58f7c632-2d25-4cb1-803a-23d73503f6c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:19:59.160783 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.160758 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58f7c632-2d25-4cb1-803a-23d73503f6c9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" (UID: \"58f7c632-2d25-4cb1-803a-23d73503f6c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:19:59.171316 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.171281 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hmz4\" (UniqueName: \"kubernetes.io/projected/58f7c632-2d25-4cb1-803a-23d73503f6c9-kube-api-access-9hmz4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" (UID: \"58f7c632-2d25-4cb1-803a-23d73503f6c9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:19:59.303680 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.303591 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:19:59.612655 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.612576 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w"] Jul 17 01:19:59.616240 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.616217 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w"] Jul 17 01:19:59.627358 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.627331 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27"] Jul 17 01:19:59.627543 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.627512 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" podUID="db15149f-4284-4d16-a7f6-39b40037856f" containerName="manager" containerID="cri-o://35f6ca3d2bfd978a8cb1919bd4a2979481d8ca656562e6eac71c3acd8ac51846" gracePeriod=2 Jul 17 01:19:59.636844 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.635758 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q"] Jul 17 01:19:59.640874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.640849 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:19:59.640981 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.640897 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27"] Jul 17 01:19:59.648349 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:19:59.648307 2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"db15149f-4284-4d16-a7f6-39b40037856f\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-67d27\": pods \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:59.651528 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:19:59.651296 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="db15149f-4284-4d16-a7f6-39b40037856f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:59.652516 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.652491 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk"] Jul 17 01:19:59.654238 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.653063 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db15149f-4284-4d16-a7f6-39b40037856f" containerName="manager" Jul 17 01:19:59.654238 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.653083 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="db15149f-4284-4d16-a7f6-39b40037856f" containerName="manager" Jul 17 01:19:59.654238 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.653192 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="db15149f-4284-4d16-a7f6-39b40037856f" containerName="manager" Jul 17 01:19:59.658133 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.657597 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd"] Jul 17 01:19:59.658133 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.657770 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:19:59.658133 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.657792 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" containerName="manager" containerID="cri-o://6f1633daa973ee09f550e8bfb146d51592db85383080b451a7b784e2163bcfbf" gracePeriod=2 Jul 17 01:19:59.658133 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.658012 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q"] Jul 17 01:19:59.660522 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.660500 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd"] Jul 17 01:19:59.668543 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.668515 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk"] Jul 17 01:19:59.676453 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.676428 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt"] Jul 17 01:19:59.676847 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.676830 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" containerName="manager" Jul 17 01:19:59.676847 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.676850 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" containerName="manager" Jul 17 01:19:59.676989 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.676933 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" containerName="manager" Jul 17 01:19:59.681355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.681332 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" Jul 17 01:19:59.689372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.689350 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt"] Jul 17 01:19:59.705527 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:19:59.705486 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="db15149f-4284-4d16-a7f6-39b40037856f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:59.707447 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:19:59.707402 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:19:59.726311 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:19:59.726270 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="db15149f-4284-4d16-a7f6-39b40037856f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:19:59.728478 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:19:59.728442 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:19:59.767316 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.767280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wrv6q\" (UID: \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:19:59.767479 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.767360 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw6p2\" (UniqueName: \"kubernetes.io/projected/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-kube-api-access-tw6p2\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wrv6q\" (UID: \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:19:59.767479 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.767438 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z22hj\" (UniqueName: \"kubernetes.io/projected/a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0-kube-api-access-z22hj\") pod \"limitador-operator-controller-manager-859d7b78bd-bvjxt\" (UID: \"a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" Jul 17 01:19:59.767479 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.767470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxf6k\" (UniqueName: \"kubernetes.io/projected/8a4b8403-f3d7-441c-89d9-1878c4862e58-kube-api-access-fxf6k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tpxhk\" (UID: \"8a4b8403-f3d7-441c-89d9-1878c4862e58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:19:59.767658 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.767524 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a4b8403-f3d7-441c-89d9-1878c4862e58-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tpxhk\" (UID: \"8a4b8403-f3d7-441c-89d9-1878c4862e58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:19:59.868429 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.868347 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw6p2\" (UniqueName: \"kubernetes.io/projected/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-kube-api-access-tw6p2\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wrv6q\" (UID: \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:19:59.868429 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.868418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z22hj\" (UniqueName: \"kubernetes.io/projected/a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0-kube-api-access-z22hj\") pod \"limitador-operator-controller-manager-859d7b78bd-bvjxt\" (UID: \"a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" Jul 17 01:19:59.868650 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.868448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxf6k\" (UniqueName: \"kubernetes.io/projected/8a4b8403-f3d7-441c-89d9-1878c4862e58-kube-api-access-fxf6k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tpxhk\" (UID: \"8a4b8403-f3d7-441c-89d9-1878c4862e58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:19:59.868650 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.868502 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a4b8403-f3d7-441c-89d9-1878c4862e58-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tpxhk\" (UID: \"8a4b8403-f3d7-441c-89d9-1878c4862e58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:19:59.868650 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.868555 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wrv6q\" (UID: \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:19:59.869048 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.869008 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a4b8403-f3d7-441c-89d9-1878c4862e58-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tpxhk\" (UID: \"8a4b8403-f3d7-441c-89d9-1878c4862e58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:19:59.869208 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.869050 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wrv6q\" (UID: \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:19:59.885019 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.884969 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z22hj\" (UniqueName: \"kubernetes.io/projected/a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0-kube-api-access-z22hj\") pod \"limitador-operator-controller-manager-859d7b78bd-bvjxt\" (UID: \"a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" Jul 17 01:19:59.885203 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.885182 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw6p2\" (UniqueName: \"kubernetes.io/projected/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-kube-api-access-tw6p2\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wrv6q\" (UID: \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:19:59.885505 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:19:59.885481 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxf6k\" (UniqueName: \"kubernetes.io/projected/8a4b8403-f3d7-441c-89d9-1878c4862e58-kube-api-access-fxf6k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tpxhk\" (UID: \"8a4b8403-f3d7-441c-89d9-1878c4862e58\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:20:00.030952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:00.030908 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:20:00.040058 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:00.040024 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:20:00.050199 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:00.050142 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" Jul 17 01:20:11.085988 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.085951 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78c566b9c6-tl9mf"] Jul 17 01:20:11.150401 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.150360 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c566b9c6-tl9mf"] Jul 17 01:20:11.150588 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.150491 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.283933 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.283889 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4jc2\" (UniqueName: \"kubernetes.io/projected/53d13840-56f9-4214-a14b-05f02ca8a795-kube-api-access-r4jc2\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.284130 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.283943 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-oauth-serving-cert\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.284130 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.283972 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-trusted-ca-bundle\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.284130 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.284028 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-service-ca\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.284130 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.284099 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-console-config\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.284356 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.284201 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/53d13840-56f9-4214-a14b-05f02ca8a795-console-serving-cert\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.284356 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.284251 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/53d13840-56f9-4214-a14b-05f02ca8a795-console-oauth-config\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.385398 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.385303 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-service-ca\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.385398 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.385343 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-console-config\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.385614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.385491 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/53d13840-56f9-4214-a14b-05f02ca8a795-console-serving-cert\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.385614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.385545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/53d13840-56f9-4214-a14b-05f02ca8a795-console-oauth-config\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.385728 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.385611 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4jc2\" (UniqueName: \"kubernetes.io/projected/53d13840-56f9-4214-a14b-05f02ca8a795-kube-api-access-r4jc2\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.385728 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.385654 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-oauth-serving-cert\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.385728 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.385685 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-trusted-ca-bundle\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.386135 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.386110 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-console-config\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.386467 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.386445 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-oauth-serving-cert\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.386559 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.386502 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-service-ca\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.386610 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.386576 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53d13840-56f9-4214-a14b-05f02ca8a795-trusted-ca-bundle\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.388060 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.388041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/53d13840-56f9-4214-a14b-05f02ca8a795-console-oauth-config\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.388482 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.388466 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/53d13840-56f9-4214-a14b-05f02ca8a795-console-serving-cert\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.393759 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.393736 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4jc2\" (UniqueName: \"kubernetes.io/projected/53d13840-56f9-4214-a14b-05f02ca8a795-kube-api-access-r4jc2\") pod \"console-78c566b9c6-tl9mf\" (UID: \"53d13840-56f9-4214-a14b-05f02ca8a795\") " pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:11.459981 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:11.459938 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:12.073186 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.071451 2564 generic.go:356] "Generic (PLEG): container finished" podID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" containerID="6f1633daa973ee09f550e8bfb146d51592db85383080b451a7b784e2163bcfbf" exitCode=0 Jul 17 01:20:12.088984 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.088789 2564 generic.go:356] "Generic (PLEG): container finished" podID="db15149f-4284-4d16-a7f6-39b40037856f" containerID="35f6ca3d2bfd978a8cb1919bd4a2979481d8ca656562e6eac71c3acd8ac51846" exitCode=0 Jul 17 01:20:12.170330 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.168585 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:20:12.171930 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:12.171497 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:20:12.173338 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.173316 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:20:12.175715 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:12.175660 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:20:12.180109 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:12.180071 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="db15149f-4284-4d16-a7f6-39b40037856f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:20:12.198640 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.198495 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt"] Jul 17 01:20:12.200299 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:20:12.200240 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda016c5c1_85d6_4dcd_b9c1_d47f0fe319d0.slice/crio-dda598a76947c779f238a1c68063c40f48f8aabe1ad85aaed0e34d9d4a4b0a23 WatchSource:0}: Error finding container dda598a76947c779f238a1c68063c40f48f8aabe1ad85aaed0e34d9d4a4b0a23: Status 404 returned error can't find the container with id dda598a76947c779f238a1c68063c40f48f8aabe1ad85aaed0e34d9d4a4b0a23 Jul 17 01:20:12.231489 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.231440 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk"] Jul 17 01:20:12.258305 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.258280 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q"] Jul 17 01:20:12.261402 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:20:12.261375 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef199419_d9fe_4e9e_b6e0_c5a2ddbddc9b.slice/crio-1a1057e8fdd2340d3ad8edeb49de79f87cdaa5f789b851c2c143d929f202ea34 WatchSource:0}: Error finding container 1a1057e8fdd2340d3ad8edeb49de79f87cdaa5f789b851c2c143d929f202ea34: Status 404 returned error can't find the container with id 1a1057e8fdd2340d3ad8edeb49de79f87cdaa5f789b851c2c143d929f202ea34 Jul 17 01:20:12.280091 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.280063 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c566b9c6-tl9mf"] Jul 17 01:20:12.283828 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:20:12.283803 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d13840_56f9_4214_a14b_05f02ca8a795.slice/crio-f683494438e45e8ee685f1b05215102dbbfdb14f41ae5fb783fee7b97ba0e3ab WatchSource:0}: Error finding container f683494438e45e8ee685f1b05215102dbbfdb14f41ae5fb783fee7b97ba0e3ab: Status 404 returned error can't find the container with id f683494438e45e8ee685f1b05215102dbbfdb14f41ae5fb783fee7b97ba0e3ab Jul 17 01:20:12.293911 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.293887 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mlqlb\" (UniqueName: \"kubernetes.io/projected/db15149f-4284-4d16-a7f6-39b40037856f-kube-api-access-mlqlb\") pod \"db15149f-4284-4d16-a7f6-39b40037856f\" (UID: \"db15149f-4284-4d16-a7f6-39b40037856f\") " Jul 17 01:20:12.294006 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.293940 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db15149f-4284-4d16-a7f6-39b40037856f-extensions-socket-volume\") pod \"db15149f-4284-4d16-a7f6-39b40037856f\" (UID: \"db15149f-4284-4d16-a7f6-39b40037856f\") " Jul 17 01:20:12.294006 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.293972 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2sw8\" (UniqueName: \"kubernetes.io/projected/8ca3dbca-df84-4262-b37b-536f4abf5e5d-kube-api-access-n2sw8\") pod \"8ca3dbca-df84-4262-b37b-536f4abf5e5d\" (UID: \"8ca3dbca-df84-4262-b37b-536f4abf5e5d\") " Jul 17 01:20:12.294439 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.294408 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db15149f-4284-4d16-a7f6-39b40037856f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "db15149f-4284-4d16-a7f6-39b40037856f" (UID: "db15149f-4284-4d16-a7f6-39b40037856f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:20:12.296480 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.296451 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ca3dbca-df84-4262-b37b-536f4abf5e5d-kube-api-access-n2sw8" (OuterVolumeSpecName: "kube-api-access-n2sw8") pod "8ca3dbca-df84-4262-b37b-536f4abf5e5d" (UID: "8ca3dbca-df84-4262-b37b-536f4abf5e5d"). InnerVolumeSpecName "kube-api-access-n2sw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:20:12.296686 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.296658 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db15149f-4284-4d16-a7f6-39b40037856f-kube-api-access-mlqlb" (OuterVolumeSpecName: "kube-api-access-mlqlb") pod "db15149f-4284-4d16-a7f6-39b40037856f" (UID: "db15149f-4284-4d16-a7f6-39b40037856f"). InnerVolumeSpecName "kube-api-access-mlqlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:20:12.395285 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.395256 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2sw8\" (UniqueName: \"kubernetes.io/projected/8ca3dbca-df84-4262-b37b-536f4abf5e5d-kube-api-access-n2sw8\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:12.395285 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.395283 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlqlb\" (UniqueName: \"kubernetes.io/projected/db15149f-4284-4d16-a7f6-39b40037856f-kube-api-access-mlqlb\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:12.395285 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:12.395294 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db15149f-4284-4d16-a7f6-39b40037856f-extensions-socket-volume\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:13.096857 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.096820 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c566b9c6-tl9mf" event={"ID":"53d13840-56f9-4214-a14b-05f02ca8a795","Type":"ContainerStarted","Data":"60ab64db5d4fc008d2b7f666760f1a5265bc846a307faa662e4af20e784d9967"} Jul 17 01:20:13.097476 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.096877 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c566b9c6-tl9mf" event={"ID":"53d13840-56f9-4214-a14b-05f02ca8a795","Type":"ContainerStarted","Data":"f683494438e45e8ee685f1b05215102dbbfdb14f41ae5fb783fee7b97ba0e3ab"} Jul 17 01:20:13.098208 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.098182 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:20:13.098208 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.098205 2564 scope.go:117] "RemoveContainer" containerID="6f1633daa973ee09f550e8bfb146d51592db85383080b451a7b784e2163bcfbf" Jul 17 01:20:13.099572 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:13.099356 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="db15149f-4284-4d16-a7f6-39b40037856f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:20:13.099652 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.099576 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:20:13.101493 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.101462 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" event={"ID":"8a4b8403-f3d7-441c-89d9-1878c4862e58","Type":"ContainerStarted","Data":"cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6"} Jul 17 01:20:13.101603 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.101506 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" event={"ID":"8a4b8403-f3d7-441c-89d9-1878c4862e58","Type":"ContainerStarted","Data":"3b362ca799721514af6bc370b8b2fbf63aa5bd131035a98c955146a06303470e"} Jul 17 01:20:13.101603 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.101595 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:20:13.103594 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.103535 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" event={"ID":"8f55e9a4-88bb-45fe-99c3-7e073d28f7c1","Type":"ContainerStarted","Data":"8655062bcd0718954b779ac1add9baed1328f94ca881f4ea9f56fd814e676c4a"} Jul 17 01:20:13.107705 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.105361 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" event={"ID":"a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0","Type":"ContainerStarted","Data":"23fefa15491fb63c2d73991932cf6ce823f193340cd8fa6014344f005378fc5a"} Jul 17 01:20:13.107705 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.105405 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" event={"ID":"a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0","Type":"ContainerStarted","Data":"dda598a76947c779f238a1c68063c40f48f8aabe1ad85aaed0e34d9d4a4b0a23"} Jul 17 01:20:13.107705 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.105788 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" Jul 17 01:20:13.109943 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.109306 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" event={"ID":"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b","Type":"ContainerStarted","Data":"d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5"} Jul 17 01:20:13.109943 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.109352 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" event={"ID":"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b","Type":"ContainerStarted","Data":"1a1057e8fdd2340d3ad8edeb49de79f87cdaa5f789b851c2c143d929f202ea34"} Jul 17 01:20:13.109943 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.109788 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:20:13.109943 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.109875 2564 scope.go:117] "RemoveContainer" containerID="35f6ca3d2bfd978a8cb1919bd4a2979481d8ca656562e6eac71c3acd8ac51846" Jul 17 01:20:13.119262 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:13.119235 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:20:13.120362 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.120316 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78c566b9c6-tl9mf" podStartSLOduration=2.120298834 podStartE2EDuration="2.120298834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:20:13.116938224 +0000 UTC m=+379.494862632" watchObservedRunningTime="2026-07-17 01:20:13.120298834 +0000 UTC m=+379.498223245" Jul 17 01:20:13.138737 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.138690 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-g7cx6" podStartSLOduration=1.7903058330000001 podStartE2EDuration="24.138675009s" totalImagesPullingTime="22.348369176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:19:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:20:13.136976325 +0000 UTC m=+379.514900735" watchObservedRunningTime="2026-07-17 01:20:13.138675009 +0000 UTC m=+379.516599467" Jul 17 01:20:13.158769 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.158712 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" podStartSLOduration=14.15869674 podStartE2EDuration="14.15869674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:20:13.156584008 +0000 UTC m=+379.534508417" watchObservedRunningTime="2026-07-17 01:20:13.15869674 +0000 UTC m=+379.536621145" Jul 17 01:20:13.172017 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.171969 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" podStartSLOduration=14.17194849 podStartE2EDuration="14.17194849s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:20:13.170945629 +0000 UTC m=+379.548870038" watchObservedRunningTime="2026-07-17 01:20:13.17194849 +0000 UTC m=+379.549872898" Jul 17 01:20:13.173099 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:13.173062 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="db15149f-4284-4d16-a7f6-39b40037856f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:20:13.194319 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:13.194262 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" podStartSLOduration=14.194243645 podStartE2EDuration="14.194243645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:20:13.192041971 +0000 UTC m=+379.569966379" watchObservedRunningTime="2026-07-17 01:20:13.194243645 +0000 UTC m=+379.572168052" Jul 17 01:20:13.194506 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:13.194405 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:20:14.132826 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:14.132789 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-67d27\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="db15149f-4284-4d16-a7f6-39b40037856f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-67d27" Jul 17 01:20:14.133452 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:14.133430 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" path="/var/lib/kubelet/pods/8ca3dbca-df84-4262-b37b-536f4abf5e5d/volumes" Jul 17 01:20:14.133873 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:14.133846 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db15149f-4284-4d16-a7f6-39b40037856f" path="/var/lib/kubelet/pods/db15149f-4284-4d16-a7f6-39b40037856f/volumes" Jul 17 01:20:14.134711 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:14.134686 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-6hkbd\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="8ca3dbca-df84-4262-b37b-536f4abf5e5d" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-6hkbd" Jul 17 01:20:14.535657 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:14.535615 2564 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-w4v6w_kuadrant-system_58f7c632-2d25-4cb1-803a-23d73503f6c9_0(beb097b82cdf8e3d43ce529fa506483d3b128667406f8d9be3bebc72a481bfe5): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-w4v6w to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 17 01:20:14.535735 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:14.535682 2564 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-w4v6w_kuadrant-system_58f7c632-2d25-4cb1-803a-23d73503f6c9_0(beb097b82cdf8e3d43ce529fa506483d3b128667406f8d9be3bebc72a481bfe5): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-w4v6w to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:20:15.122031 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:15.122001 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:20:15.124611 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:15.124578 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="58f7c632-2d25-4cb1-803a-23d73503f6c9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:20:15.128762 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:15.128735 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:20:15.131093 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:15.131064 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="58f7c632-2d25-4cb1-803a-23d73503f6c9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:20:15.219649 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:15.219608 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hmz4\" (UniqueName: \"kubernetes.io/projected/58f7c632-2d25-4cb1-803a-23d73503f6c9-kube-api-access-9hmz4\") pod \"58f7c632-2d25-4cb1-803a-23d73503f6c9\" (UID: \"58f7c632-2d25-4cb1-803a-23d73503f6c9\") " Jul 17 01:20:15.219649 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:15.219657 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58f7c632-2d25-4cb1-803a-23d73503f6c9-extensions-socket-volume\") pod \"58f7c632-2d25-4cb1-803a-23d73503f6c9\" (UID: \"58f7c632-2d25-4cb1-803a-23d73503f6c9\") " Jul 17 01:20:15.220245 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:15.219914 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58f7c632-2d25-4cb1-803a-23d73503f6c9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "58f7c632-2d25-4cb1-803a-23d73503f6c9" (UID: "58f7c632-2d25-4cb1-803a-23d73503f6c9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:20:15.220245 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:15.220007 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/58f7c632-2d25-4cb1-803a-23d73503f6c9-extensions-socket-volume\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:15.221936 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:15.221913 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58f7c632-2d25-4cb1-803a-23d73503f6c9-kube-api-access-9hmz4" (OuterVolumeSpecName: "kube-api-access-9hmz4") pod "58f7c632-2d25-4cb1-803a-23d73503f6c9" (UID: "58f7c632-2d25-4cb1-803a-23d73503f6c9"). InnerVolumeSpecName "kube-api-access-9hmz4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:20:15.321027 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:15.320989 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hmz4\" (UniqueName: \"kubernetes.io/projected/58f7c632-2d25-4cb1-803a-23d73503f6c9-kube-api-access-9hmz4\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:16.129106 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:16.129069 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:20:16.131754 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:16.131717 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="58f7c632-2d25-4cb1-803a-23d73503f6c9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:20:16.133401 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:16.133375 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58f7c632-2d25-4cb1-803a-23d73503f6c9" path="/var/lib/kubelet/pods/58f7c632-2d25-4cb1-803a-23d73503f6c9/volumes" Jul 17 01:20:16.138578 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:16.138552 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-w4v6w\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="58f7c632-2d25-4cb1-803a-23d73503f6c9" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-w4v6w" Jul 17 01:20:21.460330 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:21.460290 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:21.460822 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:21.460365 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:21.464910 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:21.464884 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:22.164247 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:22.164218 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78c566b9c6-tl9mf" Jul 17 01:20:22.214610 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:22.214579 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d47664467-9h5tk"] Jul 17 01:20:24.119235 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:24.119207 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:20:25.124220 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.124188 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:20:25.124659 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.124275 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bvjxt" Jul 17 01:20:25.179089 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.179055 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk"] Jul 17 01:20:25.179380 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.179350 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" podUID="8a4b8403-f3d7-441c-89d9-1878c4862e58" containerName="manager" containerID="cri-o://cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6" gracePeriod=10 Jul 17 01:20:25.442825 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.442794 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:20:25.493427 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.493399 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxf6k\" (UniqueName: \"kubernetes.io/projected/8a4b8403-f3d7-441c-89d9-1878c4862e58-kube-api-access-fxf6k\") pod \"8a4b8403-f3d7-441c-89d9-1878c4862e58\" (UID: \"8a4b8403-f3d7-441c-89d9-1878c4862e58\") " Jul 17 01:20:25.493611 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.493444 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a4b8403-f3d7-441c-89d9-1878c4862e58-extensions-socket-volume\") pod \"8a4b8403-f3d7-441c-89d9-1878c4862e58\" (UID: \"8a4b8403-f3d7-441c-89d9-1878c4862e58\") " Jul 17 01:20:25.493857 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.493831 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a4b8403-f3d7-441c-89d9-1878c4862e58-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8a4b8403-f3d7-441c-89d9-1878c4862e58" (UID: "8a4b8403-f3d7-441c-89d9-1878c4862e58"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:20:25.495801 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.495767 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a4b8403-f3d7-441c-89d9-1878c4862e58-kube-api-access-fxf6k" (OuterVolumeSpecName: "kube-api-access-fxf6k") pod "8a4b8403-f3d7-441c-89d9-1878c4862e58" (UID: "8a4b8403-f3d7-441c-89d9-1878c4862e58"). InnerVolumeSpecName "kube-api-access-fxf6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:20:25.504719 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.504695 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r"] Jul 17 01:20:25.505040 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.505029 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a4b8403-f3d7-441c-89d9-1878c4862e58" containerName="manager" Jul 17 01:20:25.505086 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.505042 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a4b8403-f3d7-441c-89d9-1878c4862e58" containerName="manager" Jul 17 01:20:25.505133 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.505123 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a4b8403-f3d7-441c-89d9-1878c4862e58" containerName="manager" Jul 17 01:20:25.511508 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.511491 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:25.522011 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.521986 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r"] Jul 17 01:20:25.594729 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.594695 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55382604-6ad8-426a-8658-17794a5790ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h5m9r\" (UID: \"55382604-6ad8-426a-8658-17794a5790ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:25.594908 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.594754 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clf86\" (UniqueName: \"kubernetes.io/projected/55382604-6ad8-426a-8658-17794a5790ef-kube-api-access-clf86\") pod \"kuadrant-operator-controller-manager-86878c769f-h5m9r\" (UID: \"55382604-6ad8-426a-8658-17794a5790ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:25.594908 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.594812 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxf6k\" (UniqueName: \"kubernetes.io/projected/8a4b8403-f3d7-441c-89d9-1878c4862e58-kube-api-access-fxf6k\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:25.594908 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.594830 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8a4b8403-f3d7-441c-89d9-1878c4862e58-extensions-socket-volume\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:25.695669 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.695631 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55382604-6ad8-426a-8658-17794a5790ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h5m9r\" (UID: \"55382604-6ad8-426a-8658-17794a5790ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:25.695860 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.695689 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clf86\" (UniqueName: \"kubernetes.io/projected/55382604-6ad8-426a-8658-17794a5790ef-kube-api-access-clf86\") pod \"kuadrant-operator-controller-manager-86878c769f-h5m9r\" (UID: \"55382604-6ad8-426a-8658-17794a5790ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:25.696014 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.695993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55382604-6ad8-426a-8658-17794a5790ef-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-h5m9r\" (UID: \"55382604-6ad8-426a-8658-17794a5790ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:25.706810 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.706758 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clf86\" (UniqueName: \"kubernetes.io/projected/55382604-6ad8-426a-8658-17794a5790ef-kube-api-access-clf86\") pod \"kuadrant-operator-controller-manager-86878c769f-h5m9r\" (UID: \"55382604-6ad8-426a-8658-17794a5790ef\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:25.821391 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.821350 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:25.964422 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:25.964395 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r"] Jul 17 01:20:25.965905 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:20:25.965879 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55382604_6ad8_426a_8658_17794a5790ef.slice/crio-645a136ccafb4389a634935a87df746ad298c52e048a38490f9915622eb04233 WatchSource:0}: Error finding container 645a136ccafb4389a634935a87df746ad298c52e048a38490f9915622eb04233: Status 404 returned error can't find the container with id 645a136ccafb4389a634935a87df746ad298c52e048a38490f9915622eb04233 Jul 17 01:20:26.187467 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.187428 2564 generic.go:356] "Generic (PLEG): container finished" podID="8a4b8403-f3d7-441c-89d9-1878c4862e58" containerID="cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6" exitCode=0 Jul 17 01:20:26.187916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.187473 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" Jul 17 01:20:26.187916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.187512 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" event={"ID":"8a4b8403-f3d7-441c-89d9-1878c4862e58","Type":"ContainerDied","Data":"cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6"} Jul 17 01:20:26.187916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.187557 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk" event={"ID":"8a4b8403-f3d7-441c-89d9-1878c4862e58","Type":"ContainerDied","Data":"3b362ca799721514af6bc370b8b2fbf63aa5bd131035a98c955146a06303470e"} Jul 17 01:20:26.187916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.187585 2564 scope.go:117] "RemoveContainer" containerID="cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6" Jul 17 01:20:26.189457 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.189418 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" event={"ID":"55382604-6ad8-426a-8658-17794a5790ef","Type":"ContainerStarted","Data":"fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5"} Jul 17 01:20:26.189565 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.189533 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" event={"ID":"55382604-6ad8-426a-8658-17794a5790ef","Type":"ContainerStarted","Data":"645a136ccafb4389a634935a87df746ad298c52e048a38490f9915622eb04233"} Jul 17 01:20:26.189641 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.189567 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:26.196917 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.196901 2564 scope.go:117] "RemoveContainer" containerID="cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6" Jul 17 01:20:26.197206 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:26.197186 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6\": container with ID starting with cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6 not found: ID does not exist" containerID="cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6" Jul 17 01:20:26.197247 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.197213 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6"} err="failed to get container status \"cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6\": rpc error: code = NotFound desc = could not find container \"cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6\": container with ID starting with cc5229743a6c5cccbfbc275f355be4f01d0e7cf120be79dd41ddc4d1cc3308a6 not found: ID does not exist" Jul 17 01:20:26.215275 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.215204 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk"] Jul 17 01:20:26.221030 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.221001 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tpxhk"] Jul 17 01:20:26.236695 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:26.236650 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" podStartSLOduration=1.236635277 podStartE2EDuration="1.236635277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:20:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:20:26.234546377 +0000 UTC m=+392.612470785" watchObservedRunningTime="2026-07-17 01:20:26.236635277 +0000 UTC m=+392.614559685" Jul 17 01:20:28.133387 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:28.133355 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a4b8403-f3d7-441c-89d9-1878c4862e58" path="/var/lib/kubelet/pods/8a4b8403-f3d7-441c-89d9-1878c4862e58/volumes" Jul 17 01:20:37.199080 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:37.199049 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:20:37.249525 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:37.249485 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q"] Jul 17 01:20:37.249789 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:37.249763 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" podUID="ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b" containerName="manager" containerID="cri-o://d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5" gracePeriod=10 Jul 17 01:20:38.038470 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.038436 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:20:38.109351 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.109267 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw6p2\" (UniqueName: \"kubernetes.io/projected/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-kube-api-access-tw6p2\") pod \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\" (UID: \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\") " Jul 17 01:20:38.109351 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.109321 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-extensions-socket-volume\") pod \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\" (UID: \"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b\") " Jul 17 01:20:38.109726 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.109693 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b" (UID: "ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:20:38.111579 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.111536 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-kube-api-access-tw6p2" (OuterVolumeSpecName: "kube-api-access-tw6p2") pod "ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b" (UID: "ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b"). InnerVolumeSpecName "kube-api-access-tw6p2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:20:38.210322 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.210281 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw6p2\" (UniqueName: \"kubernetes.io/projected/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-kube-api-access-tw6p2\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:38.210322 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.210314 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b-extensions-socket-volume\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:38.261013 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.260981 2564 generic.go:356] "Generic (PLEG): container finished" podID="ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b" containerID="d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5" exitCode=0 Jul 17 01:20:38.261194 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.261037 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" Jul 17 01:20:38.261194 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.261048 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" event={"ID":"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b","Type":"ContainerDied","Data":"d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5"} Jul 17 01:20:38.261194 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.261104 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q" event={"ID":"ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b","Type":"ContainerDied","Data":"1a1057e8fdd2340d3ad8edeb49de79f87cdaa5f789b851c2c143d929f202ea34"} Jul 17 01:20:38.261194 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.261128 2564 scope.go:117] "RemoveContainer" containerID="d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5" Jul 17 01:20:38.270752 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.270732 2564 scope.go:117] "RemoveContainer" containerID="d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5" Jul 17 01:20:38.271028 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:38.271008 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5\": container with ID starting with d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5 not found: ID does not exist" containerID="d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5" Jul 17 01:20:38.271099 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.271041 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5"} err="failed to get container status \"d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5\": rpc error: code = NotFound desc = could not find container \"d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5\": container with ID starting with d21395b2423ce98aee7fd85a003767270dedffebfa44a4ba8a81b8e1271086f5 not found: ID does not exist" Jul 17 01:20:38.278550 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.278527 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q"] Jul 17 01:20:38.284563 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:38.284542 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wrv6q"] Jul 17 01:20:40.133802 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:40.133765 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b" path="/var/lib/kubelet/pods/ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b/volumes" Jul 17 01:20:41.474665 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.474630 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn"] Jul 17 01:20:41.475127 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.475111 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b" containerName="manager" Jul 17 01:20:41.475202 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.475129 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b" containerName="manager" Jul 17 01:20:41.475202 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.475197 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef199419-d9fe-4e9e-b6e0-c5a2ddbddc9b" containerName="manager" Jul 17 01:20:41.787809 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.787715 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn"] Jul 17 01:20:41.787809 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.787736 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.790954 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.790934 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 17 01:20:41.791039 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.790934 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-krlwb\"" Jul 17 01:20:41.840965 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.840925 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.840965 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.840962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.841224 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.840995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.841224 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.841057 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.841224 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.841108 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.841224 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.841126 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.841224 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.841153 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.841408 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.841236 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjtf2\" (UniqueName: \"kubernetes.io/projected/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-kube-api-access-jjtf2\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.841408 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.841273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942379 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942341 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942546 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942546 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942408 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942546 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942546 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942467 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942546 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942495 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942546 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942541 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjtf2\" (UniqueName: \"kubernetes.io/projected/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-kube-api-access-jjtf2\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942841 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942597 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942841 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942641 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942841 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.942841 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.942821 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.943056 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.943030 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.943135 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.943114 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.943226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.943210 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.945020 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.944999 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.945280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.945259 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.951406 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.951377 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:41.951718 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:41.951700 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjtf2\" (UniqueName: \"kubernetes.io/projected/d4ac076b-8b5c-4dc1-82db-6fa21f323ecf-kube-api-access-jjtf2\") pod \"maas-default-gateway-openshift-default-56654dcf7-dxtsn\" (UID: \"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:42.098093 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:42.098006 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:42.241674 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:42.241647 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn"] Jul 17 01:20:42.243377 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:20:42.243345 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4ac076b_8b5c_4dc1_82db_6fa21f323ecf.slice/crio-370b5f5405e19b92fe6a63c1f184d6ad21b6b8fd9b75608d10d6b989cf7da7cf WatchSource:0}: Error finding container 370b5f5405e19b92fe6a63c1f184d6ad21b6b8fd9b75608d10d6b989cf7da7cf: Status 404 returned error can't find the container with id 370b5f5405e19b92fe6a63c1f184d6ad21b6b8fd9b75608d10d6b989cf7da7cf Jul 17 01:20:42.291733 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:42.291699 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" event={"ID":"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf","Type":"ContainerStarted","Data":"370b5f5405e19b92fe6a63c1f184d6ad21b6b8fd9b75608d10d6b989cf7da7cf"} Jul 17 01:20:47.236414 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.236365 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d47664467-9h5tk" podUID="c2467b83-14db-4717-9e1c-3a1fb98dd39d" containerName="console" containerID="cri-o://895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456" gracePeriod=15 Jul 17 01:20:47.506220 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.506191 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:20:47.598366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.598329 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-config\") pod \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " Jul 17 01:20:47.598366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.598366 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-serving-cert\") pod \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " Jul 17 01:20:47.598617 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.598408 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-trusted-ca-bundle\") pod \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " Jul 17 01:20:47.598617 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.598475 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8ck9r\" (UniqueName: \"kubernetes.io/projected/c2467b83-14db-4717-9e1c-3a1fb98dd39d-kube-api-access-8ck9r\") pod \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " Jul 17 01:20:47.598617 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.598548 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-oauth-config\") pod \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " Jul 17 01:20:47.598617 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.598594 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-oauth-serving-cert\") pod \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " Jul 17 01:20:47.598822 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.598622 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-service-ca\") pod \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\" (UID: \"c2467b83-14db-4717-9e1c-3a1fb98dd39d\") " Jul 17 01:20:47.598898 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.598863 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-config" (OuterVolumeSpecName: "console-config") pod "c2467b83-14db-4717-9e1c-3a1fb98dd39d" (UID: "c2467b83-14db-4717-9e1c-3a1fb98dd39d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:20:47.599226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.599123 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c2467b83-14db-4717-9e1c-3a1fb98dd39d" (UID: "c2467b83-14db-4717-9e1c-3a1fb98dd39d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:20:47.599226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.599150 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-service-ca" (OuterVolumeSpecName: "service-ca") pod "c2467b83-14db-4717-9e1c-3a1fb98dd39d" (UID: "c2467b83-14db-4717-9e1c-3a1fb98dd39d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:20:47.599226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.599145 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c2467b83-14db-4717-9e1c-3a1fb98dd39d" (UID: "c2467b83-14db-4717-9e1c-3a1fb98dd39d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:20:47.601123 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.601088 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c2467b83-14db-4717-9e1c-3a1fb98dd39d" (UID: "c2467b83-14db-4717-9e1c-3a1fb98dd39d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:20:47.601244 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.601154 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2467b83-14db-4717-9e1c-3a1fb98dd39d-kube-api-access-8ck9r" (OuterVolumeSpecName: "kube-api-access-8ck9r") pod "c2467b83-14db-4717-9e1c-3a1fb98dd39d" (UID: "c2467b83-14db-4717-9e1c-3a1fb98dd39d"). InnerVolumeSpecName "kube-api-access-8ck9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:20:47.601351 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.601272 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c2467b83-14db-4717-9e1c-3a1fb98dd39d" (UID: "c2467b83-14db-4717-9e1c-3a1fb98dd39d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:20:47.700357 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.700316 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-config\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:47.700357 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.700361 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-serving-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:47.700596 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.700372 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-trusted-ca-bundle\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:47.700596 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.700381 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ck9r\" (UniqueName: \"kubernetes.io/projected/c2467b83-14db-4717-9e1c-3a1fb98dd39d-kube-api-access-8ck9r\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:47.700596 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.700395 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2467b83-14db-4717-9e1c-3a1fb98dd39d-console-oauth-config\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:47.700596 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.700409 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-oauth-serving-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:47.700596 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:47.700422 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2467b83-14db-4717-9e1c-3a1fb98dd39d-service-ca\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:20:48.333728 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.333686 2564 generic.go:356] "Generic (PLEG): container finished" podID="c2467b83-14db-4717-9e1c-3a1fb98dd39d" containerID="895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456" exitCode=0 Jul 17 01:20:48.334236 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.333731 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d47664467-9h5tk" event={"ID":"c2467b83-14db-4717-9e1c-3a1fb98dd39d","Type":"ContainerDied","Data":"895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456"} Jul 17 01:20:48.334236 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.333810 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d47664467-9h5tk" event={"ID":"c2467b83-14db-4717-9e1c-3a1fb98dd39d","Type":"ContainerDied","Data":"eb38b6858f2a2cbababdc193b14002ad878fafde92f4766d4cf528cfc3d051be"} Jul 17 01:20:48.334236 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.333822 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d47664467-9h5tk" Jul 17 01:20:48.334236 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.333841 2564 scope.go:117] "RemoveContainer" containerID="895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456" Jul 17 01:20:48.352198 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.352136 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d47664467-9h5tk"] Jul 17 01:20:48.355329 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.355302 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d47664467-9h5tk"] Jul 17 01:20:48.933536 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.933516 2564 scope.go:117] "RemoveContainer" containerID="895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456" Jul 17 01:20:48.933884 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:20:48.933859 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456\": container with ID starting with 895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456 not found: ID does not exist" containerID="895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456" Jul 17 01:20:48.933994 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.933894 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456"} err="failed to get container status \"895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456\": rpc error: code = NotFound desc = could not find container \"895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456\": container with ID starting with 895ca36fff0e2b8aaa4586117c90d70ad26a6efaf2e07a5e4a4424c3001c0456 not found: ID does not exist" Jul 17 01:20:48.964473 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.964439 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:20:48.964572 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.964507 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:20:48.964572 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:48.964537 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:20:49.341816 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:49.341729 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" event={"ID":"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf","Type":"ContainerStarted","Data":"6be70de2d016b5d5eb30d581be5e3a2f4afb5a438edd1e11f65e1e159eb30dbd"} Jul 17 01:20:49.365012 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:49.364962 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" podStartSLOduration=1.6461383330000001 podStartE2EDuration="8.36494489s" totalImagesPullingTime="6.718806557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:20:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:20:49.362865266 +0000 UTC m=+415.740789675" watchObservedRunningTime="2026-07-17 01:20:49.36494489 +0000 UTC m=+415.742869297" Jul 17 01:20:50.098806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:50.098773 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:50.103832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:50.103793 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:50.134220 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:50.134186 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2467b83-14db-4717-9e1c-3a1fb98dd39d" path="/var/lib/kubelet/pods/c2467b83-14db-4717-9e1c-3a1fb98dd39d/volumes" Jul 17 01:20:50.346972 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:50.346944 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:50.348056 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:50.348041 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:20:55.252181 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.252084 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6j95"] Jul 17 01:20:55.252536 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.252456 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2467b83-14db-4717-9e1c-3a1fb98dd39d" containerName="console" Jul 17 01:20:55.252536 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.252466 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2467b83-14db-4717-9e1c-3a1fb98dd39d" containerName="console" Jul 17 01:20:55.252610 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.252544 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2467b83-14db-4717-9e1c-3a1fb98dd39d" containerName="console" Jul 17 01:20:55.255322 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.255304 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:20:55.257699 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.257680 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 17 01:20:55.262925 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.262899 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6j95"] Jul 17 01:20:55.350590 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.350552 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6j95"] Jul 17 01:20:55.373751 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.373723 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e884513e-ad58-414b-a9f3-1e4b0f9b691d-config-file\") pod \"limitador-limitador-64c94fd64f-x6j95\" (UID: \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:20:55.373904 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.373796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lmfm\" (UniqueName: \"kubernetes.io/projected/e884513e-ad58-414b-a9f3-1e4b0f9b691d-kube-api-access-7lmfm\") pod \"limitador-limitador-64c94fd64f-x6j95\" (UID: \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:20:55.475189 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.475127 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e884513e-ad58-414b-a9f3-1e4b0f9b691d-config-file\") pod \"limitador-limitador-64c94fd64f-x6j95\" (UID: \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:20:55.475351 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.475242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lmfm\" (UniqueName: \"kubernetes.io/projected/e884513e-ad58-414b-a9f3-1e4b0f9b691d-kube-api-access-7lmfm\") pod \"limitador-limitador-64c94fd64f-x6j95\" (UID: \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:20:55.475740 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.475721 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e884513e-ad58-414b-a9f3-1e4b0f9b691d-config-file\") pod \"limitador-limitador-64c94fd64f-x6j95\" (UID: \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:20:55.482973 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.482954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lmfm\" (UniqueName: \"kubernetes.io/projected/e884513e-ad58-414b-a9f3-1e4b0f9b691d-kube-api-access-7lmfm\") pod \"limitador-limitador-64c94fd64f-x6j95\" (UID: \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:20:55.565776 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.565690 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:20:55.706079 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:55.705968 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6j95"] Jul 17 01:20:55.708308 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:20:55.708277 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode884513e_ad58_414b_a9f3_1e4b0f9b691d.slice/crio-db57503235e1efa2de8bdda01a6b1f1c54277608df2104e5e9ce63962cca02f3 WatchSource:0}: Error finding container db57503235e1efa2de8bdda01a6b1f1c54277608df2104e5e9ce63962cca02f3: Status 404 returned error can't find the container with id db57503235e1efa2de8bdda01a6b1f1c54277608df2104e5e9ce63962cca02f3 Jul 17 01:20:56.058655 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.058615 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-xmjrr"] Jul 17 01:20:56.061807 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.061784 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xmjrr" Jul 17 01:20:56.064200 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.064178 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rkrzp\"" Jul 17 01:20:56.068280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.068258 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xmjrr"] Jul 17 01:20:56.181581 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.181545 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfqsl\" (UniqueName: \"kubernetes.io/projected/d795606a-59e2-41d2-b977-e46ce5f99c9c-kube-api-access-hfqsl\") pod \"authorino-58cd955675-xmjrr\" (UID: \"d795606a-59e2-41d2-b977-e46ce5f99c9c\") " pod="kuadrant-system/authorino-58cd955675-xmjrr" Jul 17 01:20:56.208558 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.208514 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-wdjsq"] Jul 17 01:20:56.210857 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.210838 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wdjsq" Jul 17 01:20:56.217348 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.217308 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wdjsq"] Jul 17 01:20:56.282340 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.282309 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfqsl\" (UniqueName: \"kubernetes.io/projected/d795606a-59e2-41d2-b977-e46ce5f99c9c-kube-api-access-hfqsl\") pod \"authorino-58cd955675-xmjrr\" (UID: \"d795606a-59e2-41d2-b977-e46ce5f99c9c\") " pod="kuadrant-system/authorino-58cd955675-xmjrr" Jul 17 01:20:56.290831 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.290802 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfqsl\" (UniqueName: \"kubernetes.io/projected/d795606a-59e2-41d2-b977-e46ce5f99c9c-kube-api-access-hfqsl\") pod \"authorino-58cd955675-xmjrr\" (UID: \"d795606a-59e2-41d2-b977-e46ce5f99c9c\") " pod="kuadrant-system/authorino-58cd955675-xmjrr" Jul 17 01:20:56.373193 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.373080 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xmjrr" Jul 17 01:20:56.379361 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.379321 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" event={"ID":"e884513e-ad58-414b-a9f3-1e4b0f9b691d","Type":"ContainerStarted","Data":"db57503235e1efa2de8bdda01a6b1f1c54277608df2104e5e9ce63962cca02f3"} Jul 17 01:20:56.382728 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.382704 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kct9h\" (UniqueName: \"kubernetes.io/projected/d5c10ef9-83a9-4fc3-999e-c5327cc29b8e-kube-api-access-kct9h\") pod \"authorino-75d556bd77-wdjsq\" (UID: \"d5c10ef9-83a9-4fc3-999e-c5327cc29b8e\") " pod="kuadrant-system/authorino-75d556bd77-wdjsq" Jul 17 01:20:56.483972 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.483552 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kct9h\" (UniqueName: \"kubernetes.io/projected/d5c10ef9-83a9-4fc3-999e-c5327cc29b8e-kube-api-access-kct9h\") pod \"authorino-75d556bd77-wdjsq\" (UID: \"d5c10ef9-83a9-4fc3-999e-c5327cc29b8e\") " pod="kuadrant-system/authorino-75d556bd77-wdjsq" Jul 17 01:20:56.494530 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.494468 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kct9h\" (UniqueName: \"kubernetes.io/projected/d5c10ef9-83a9-4fc3-999e-c5327cc29b8e-kube-api-access-kct9h\") pod \"authorino-75d556bd77-wdjsq\" (UID: \"d5c10ef9-83a9-4fc3-999e-c5327cc29b8e\") " pod="kuadrant-system/authorino-75d556bd77-wdjsq" Jul 17 01:20:56.521394 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.521360 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wdjsq" Jul 17 01:20:56.527130 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.526089 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-xmjrr"] Jul 17 01:20:56.535842 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:20:56.535812 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd795606a_59e2_41d2_b977_e46ce5f99c9c.slice/crio-81e177bb447662c63917977b011bc7917ae8fc97048c4a66971ca22f09d3ff2b WatchSource:0}: Error finding container 81e177bb447662c63917977b011bc7917ae8fc97048c4a66971ca22f09d3ff2b: Status 404 returned error can't find the container with id 81e177bb447662c63917977b011bc7917ae8fc97048c4a66971ca22f09d3ff2b Jul 17 01:20:56.711137 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:56.711102 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wdjsq"] Jul 17 01:20:56.716138 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:20:56.716099 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5c10ef9_83a9_4fc3_999e_c5327cc29b8e.slice/crio-e4bde07d349cfc13e7a629e217e4bea110116b7fed427c7c3e7bff542a68a3f7 WatchSource:0}: Error finding container e4bde07d349cfc13e7a629e217e4bea110116b7fed427c7c3e7bff542a68a3f7: Status 404 returned error can't find the container with id e4bde07d349cfc13e7a629e217e4bea110116b7fed427c7c3e7bff542a68a3f7 Jul 17 01:20:57.404839 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:57.404772 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xmjrr" event={"ID":"d795606a-59e2-41d2-b977-e46ce5f99c9c","Type":"ContainerStarted","Data":"81e177bb447662c63917977b011bc7917ae8fc97048c4a66971ca22f09d3ff2b"} Jul 17 01:20:57.409544 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:20:57.409465 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wdjsq" event={"ID":"d5c10ef9-83a9-4fc3-999e-c5327cc29b8e","Type":"ContainerStarted","Data":"e4bde07d349cfc13e7a629e217e4bea110116b7fed427c7c3e7bff542a68a3f7"} Jul 17 01:21:01.437682 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:01.437640 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xmjrr" event={"ID":"d795606a-59e2-41d2-b977-e46ce5f99c9c","Type":"ContainerStarted","Data":"58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede"} Jul 17 01:21:01.439105 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:01.439079 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wdjsq" event={"ID":"d5c10ef9-83a9-4fc3-999e-c5327cc29b8e","Type":"ContainerStarted","Data":"55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8"} Jul 17 01:21:01.440354 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:01.440331 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" event={"ID":"e884513e-ad58-414b-a9f3-1e4b0f9b691d","Type":"ContainerStarted","Data":"b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188"} Jul 17 01:21:01.440481 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:01.440466 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:21:01.452372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:01.452330 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-xmjrr" podStartSLOduration=1.102820043 podStartE2EDuration="5.452315068s" totalImagesPullingTime="4.349495025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:21:01.451197749 +0000 UTC m=+427.829122159" watchObservedRunningTime="2026-07-17 01:21:01.452315068 +0000 UTC m=+427.830239475" Jul 17 01:21:01.468647 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:01.468592 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-wdjsq" podStartSLOduration=1.304906656 podStartE2EDuration="5.468571475s" totalImagesPullingTime="4.163664819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:21:01.465066635 +0000 UTC m=+427.842991043" watchObservedRunningTime="2026-07-17 01:21:01.468571475 +0000 UTC m=+427.846495884" Jul 17 01:21:01.484307 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:01.484244 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" podStartSLOduration=1.345623173 podStartE2EDuration="6.484226322s" totalImagesPullingTime="5.138603149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:21:01.483469689 +0000 UTC m=+427.861394097" watchObservedRunningTime="2026-07-17 01:21:01.484226322 +0000 UTC m=+427.862150736" Jul 17 01:21:01.493648 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:01.493609 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xmjrr"] Jul 17 01:21:03.450823 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:03.450775 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-xmjrr" podUID="d795606a-59e2-41d2-b977-e46ce5f99c9c" containerName="authorino" containerID="cri-o://58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede" gracePeriod=30 Jul 17 01:21:03.707226 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:03.707153 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xmjrr" Jul 17 01:21:03.755725 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:03.755688 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfqsl\" (UniqueName: \"kubernetes.io/projected/d795606a-59e2-41d2-b977-e46ce5f99c9c-kube-api-access-hfqsl\") pod \"d795606a-59e2-41d2-b977-e46ce5f99c9c\" (UID: \"d795606a-59e2-41d2-b977-e46ce5f99c9c\") " Jul 17 01:21:03.757853 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:03.757831 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d795606a-59e2-41d2-b977-e46ce5f99c9c-kube-api-access-hfqsl" (OuterVolumeSpecName: "kube-api-access-hfqsl") pod "d795606a-59e2-41d2-b977-e46ce5f99c9c" (UID: "d795606a-59e2-41d2-b977-e46ce5f99c9c"). InnerVolumeSpecName "kube-api-access-hfqsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:21:03.856615 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:03.856573 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfqsl\" (UniqueName: \"kubernetes.io/projected/d795606a-59e2-41d2-b977-e46ce5f99c9c-kube-api-access-hfqsl\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:21:04.457156 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:04.457121 2564 generic.go:356] "Generic (PLEG): container finished" podID="d795606a-59e2-41d2-b977-e46ce5f99c9c" containerID="58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede" exitCode=0 Jul 17 01:21:04.457614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:04.457175 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-xmjrr" Jul 17 01:21:04.457614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:04.457195 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xmjrr" event={"ID":"d795606a-59e2-41d2-b977-e46ce5f99c9c","Type":"ContainerDied","Data":"58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede"} Jul 17 01:21:04.457614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:04.457229 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-xmjrr" event={"ID":"d795606a-59e2-41d2-b977-e46ce5f99c9c","Type":"ContainerDied","Data":"81e177bb447662c63917977b011bc7917ae8fc97048c4a66971ca22f09d3ff2b"} Jul 17 01:21:04.457614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:04.457252 2564 scope.go:117] "RemoveContainer" containerID="58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede" Jul 17 01:21:04.465893 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:04.465874 2564 scope.go:117] "RemoveContainer" containerID="58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede" Jul 17 01:21:04.466187 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:21:04.466150 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede\": container with ID starting with 58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede not found: ID does not exist" containerID="58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede" Jul 17 01:21:04.466278 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:04.466196 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede"} err="failed to get container status \"58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede\": rpc error: code = NotFound desc = could not find container \"58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede\": container with ID starting with 58495a3ec9ebd38f7b5f4d82f3bfcd311524f65f22ea4205ad7b7b9302174ede not found: ID does not exist" Jul 17 01:21:04.473954 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:04.473929 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-xmjrr"] Jul 17 01:21:04.476022 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:04.476003 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-xmjrr"] Jul 17 01:21:06.134289 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:06.134252 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d795606a-59e2-41d2-b977-e46ce5f99c9c" path="/var/lib/kubelet/pods/d795606a-59e2-41d2-b977-e46ce5f99c9c/volumes" Jul 17 01:21:10.264731 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:10.264694 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6j95"] Jul 17 01:21:10.265308 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:10.264939 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" podUID="e884513e-ad58-414b-a9f3-1e4b0f9b691d" containerName="limitador" containerID="cri-o://b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188" gracePeriod=30 Jul 17 01:21:10.265686 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:10.265650 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:21:11.215593 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.215570 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:21:11.316570 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.316532 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lmfm\" (UniqueName: \"kubernetes.io/projected/e884513e-ad58-414b-a9f3-1e4b0f9b691d-kube-api-access-7lmfm\") pod \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\" (UID: \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\") " Jul 17 01:21:11.316963 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.316620 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e884513e-ad58-414b-a9f3-1e4b0f9b691d-config-file\") pod \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\" (UID: \"e884513e-ad58-414b-a9f3-1e4b0f9b691d\") " Jul 17 01:21:11.316963 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.316945 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e884513e-ad58-414b-a9f3-1e4b0f9b691d-config-file" (OuterVolumeSpecName: "config-file") pod "e884513e-ad58-414b-a9f3-1e4b0f9b691d" (UID: "e884513e-ad58-414b-a9f3-1e4b0f9b691d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:21:11.318887 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.318866 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e884513e-ad58-414b-a9f3-1e4b0f9b691d-kube-api-access-7lmfm" (OuterVolumeSpecName: "kube-api-access-7lmfm") pod "e884513e-ad58-414b-a9f3-1e4b0f9b691d" (UID: "e884513e-ad58-414b-a9f3-1e4b0f9b691d"). InnerVolumeSpecName "kube-api-access-7lmfm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:21:11.417270 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.417237 2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e884513e-ad58-414b-a9f3-1e4b0f9b691d-config-file\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:21:11.417270 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.417268 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lmfm\" (UniqueName: \"kubernetes.io/projected/e884513e-ad58-414b-a9f3-1e4b0f9b691d-kube-api-access-7lmfm\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:21:11.498112 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.498079 2564 generic.go:356] "Generic (PLEG): container finished" podID="e884513e-ad58-414b-a9f3-1e4b0f9b691d" containerID="b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188" exitCode=0 Jul 17 01:21:11.498365 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.498125 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" Jul 17 01:21:11.498365 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.498201 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" event={"ID":"e884513e-ad58-414b-a9f3-1e4b0f9b691d","Type":"ContainerDied","Data":"b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188"} Jul 17 01:21:11.498365 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.498305 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x6j95" event={"ID":"e884513e-ad58-414b-a9f3-1e4b0f9b691d","Type":"ContainerDied","Data":"db57503235e1efa2de8bdda01a6b1f1c54277608df2104e5e9ce63962cca02f3"} Jul 17 01:21:11.498365 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.498338 2564 scope.go:117] "RemoveContainer" containerID="b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188" Jul 17 01:21:11.507365 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.507342 2564 scope.go:117] "RemoveContainer" containerID="b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188" Jul 17 01:21:11.507651 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:21:11.507633 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188\": container with ID starting with b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188 not found: ID does not exist" containerID="b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188" Jul 17 01:21:11.507703 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.507663 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188"} err="failed to get container status \"b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188\": rpc error: code = NotFound desc = could not find container \"b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188\": container with ID starting with b42b20a661c94cd88e6be228777952fc8ef1783191ab5ee1d456c0d85c3f0188 not found: ID does not exist" Jul 17 01:21:11.518867 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.518841 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6j95"] Jul 17 01:21:11.521889 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:11.521869 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x6j95"] Jul 17 01:21:12.053976 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.053936 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb"] Jul 17 01:21:12.054360 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.054345 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d795606a-59e2-41d2-b977-e46ce5f99c9c" containerName="authorino" Jul 17 01:21:12.054410 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.054361 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d795606a-59e2-41d2-b977-e46ce5f99c9c" containerName="authorino" Jul 17 01:21:12.054410 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.054371 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e884513e-ad58-414b-a9f3-1e4b0f9b691d" containerName="limitador" Jul 17 01:21:12.054410 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.054377 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e884513e-ad58-414b-a9f3-1e4b0f9b691d" containerName="limitador" Jul 17 01:21:12.054507 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.054446 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e884513e-ad58-414b-a9f3-1e4b0f9b691d" containerName="limitador" Jul 17 01:21:12.054507 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.054454 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d795606a-59e2-41d2-b977-e46ce5f99c9c" containerName="authorino" Jul 17 01:21:12.056983 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.056968 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:12.059818 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.059794 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 17 01:21:12.059945 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.059802 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 17 01:21:12.059945 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.059802 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 17 01:21:12.060935 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.060919 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-k7szf\"" Jul 17 01:21:12.065607 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.065586 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb"] Jul 17 01:21:12.122966 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.122929 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/700b1d31-8695-4938-b909-4a97974e81e5-data\") pod \"postgres-69fb4594cf-ls7lb\" (UID: \"700b1d31-8695-4938-b909-4a97974e81e5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:12.123134 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.122990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bzwc\" (UniqueName: \"kubernetes.io/projected/700b1d31-8695-4938-b909-4a97974e81e5-kube-api-access-8bzwc\") pod \"postgres-69fb4594cf-ls7lb\" (UID: \"700b1d31-8695-4938-b909-4a97974e81e5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:12.133994 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.133965 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e884513e-ad58-414b-a9f3-1e4b0f9b691d" path="/var/lib/kubelet/pods/e884513e-ad58-414b-a9f3-1e4b0f9b691d/volumes" Jul 17 01:21:12.223943 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.223909 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bzwc\" (UniqueName: \"kubernetes.io/projected/700b1d31-8695-4938-b909-4a97974e81e5-kube-api-access-8bzwc\") pod \"postgres-69fb4594cf-ls7lb\" (UID: \"700b1d31-8695-4938-b909-4a97974e81e5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:12.224124 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.223982 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/700b1d31-8695-4938-b909-4a97974e81e5-data\") pod \"postgres-69fb4594cf-ls7lb\" (UID: \"700b1d31-8695-4938-b909-4a97974e81e5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:12.224353 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.224333 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/700b1d31-8695-4938-b909-4a97974e81e5-data\") pod \"postgres-69fb4594cf-ls7lb\" (UID: \"700b1d31-8695-4938-b909-4a97974e81e5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:12.231844 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.231821 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bzwc\" (UniqueName: \"kubernetes.io/projected/700b1d31-8695-4938-b909-4a97974e81e5-kube-api-access-8bzwc\") pod \"postgres-69fb4594cf-ls7lb\" (UID: \"700b1d31-8695-4938-b909-4a97974e81e5\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:12.367446 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.367360 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:12.506195 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:12.506151 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb"] Jul 17 01:21:12.507687 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:21:12.507664 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod700b1d31_8695_4938_b909_4a97974e81e5.slice/crio-73d4efbb59af95e6ac93e0ad15da6775eb82264122d0d72e33b8cbd12aa1a993 WatchSource:0}: Error finding container 73d4efbb59af95e6ac93e0ad15da6775eb82264122d0d72e33b8cbd12aa1a993: Status 404 returned error can't find the container with id 73d4efbb59af95e6ac93e0ad15da6775eb82264122d0d72e33b8cbd12aa1a993 Jul 17 01:21:13.513932 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:13.513858 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" event={"ID":"700b1d31-8695-4938-b909-4a97974e81e5","Type":"ContainerStarted","Data":"73d4efbb59af95e6ac93e0ad15da6775eb82264122d0d72e33b8cbd12aa1a993"} Jul 17 01:21:18.546579 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:18.546536 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" event={"ID":"700b1d31-8695-4938-b909-4a97974e81e5","Type":"ContainerStarted","Data":"15d0e164db311c240a04d031325a7b71da206be010ea3edcc9efc22f9f6e4a0e"} Jul 17 01:21:18.547064 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:18.546665 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:18.562832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:18.562775 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" podStartSLOduration=1.569584243 podStartE2EDuration="6.562760443s" totalImagesPullingTime="4.9931762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:21:18.562081891 +0000 UTC m=+444.940006300" watchObservedRunningTime="2026-07-17 01:21:18.562760443 +0000 UTC m=+444.940684860" Jul 17 01:21:24.582657 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:24.582626 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ls7lb" Jul 17 01:21:25.100961 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.100923 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phv97"] Jul 17 01:21:25.110291 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.110259 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-phv97" Jul 17 01:21:25.111308 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.111281 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phv97"] Jul 17 01:21:25.247493 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.247460 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cj9b\" (UniqueName: \"kubernetes.io/projected/9d02cba3-4a3d-4262-9c1b-a85277d19ae4-kube-api-access-4cj9b\") pod \"authorino-5d4b67c446-phv97\" (UID: \"9d02cba3-4a3d-4262-9c1b-a85277d19ae4\") " pod="kuadrant-system/authorino-5d4b67c446-phv97" Jul 17 01:21:25.327047 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.327008 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phv97"] Jul 17 01:21:25.327321 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:21:25.327299 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4cj9b], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-phv97" podUID="9d02cba3-4a3d-4262-9c1b-a85277d19ae4" Jul 17 01:21:25.348741 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.348686 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cj9b\" (UniqueName: \"kubernetes.io/projected/9d02cba3-4a3d-4262-9c1b-a85277d19ae4-kube-api-access-4cj9b\") pod \"authorino-5d4b67c446-phv97\" (UID: \"9d02cba3-4a3d-4262-9c1b-a85277d19ae4\") " pod="kuadrant-system/authorino-5d4b67c446-phv97" Jul 17 01:21:25.351648 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.351582 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8988bbd85-dcst4"] Jul 17 01:21:25.355544 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.355526 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8988bbd85-dcst4" Jul 17 01:21:25.361229 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.361201 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8988bbd85-dcst4"] Jul 17 01:21:25.362021 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.361996 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cj9b\" (UniqueName: \"kubernetes.io/projected/9d02cba3-4a3d-4262-9c1b-a85277d19ae4-kube-api-access-4cj9b\") pod \"authorino-5d4b67c446-phv97\" (UID: \"9d02cba3-4a3d-4262-9c1b-a85277d19ae4\") " pod="kuadrant-system/authorino-5d4b67c446-phv97" Jul 17 01:21:25.450141 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.450103 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8vrw\" (UniqueName: \"kubernetes.io/projected/0159af74-ba53-40e0-9c67-aeb62d2e90de-kube-api-access-p8vrw\") pod \"authorino-8988bbd85-dcst4\" (UID: \"0159af74-ba53-40e0-9c67-aeb62d2e90de\") " pod="kuadrant-system/authorino-8988bbd85-dcst4" Jul 17 01:21:25.485881 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.485845 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8988bbd85-dcst4"] Jul 17 01:21:25.486144 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:21:25.486123 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-p8vrw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-8988bbd85-dcst4" podUID="0159af74-ba53-40e0-9c67-aeb62d2e90de" Jul 17 01:21:25.511619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.511590 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bb5679577-4rmnf"] Jul 17 01:21:25.515435 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.515412 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:21:25.517765 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.517746 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 17 01:21:25.521112 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.521087 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bb5679577-4rmnf"] Jul 17 01:21:25.551497 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.551460 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8vrw\" (UniqueName: \"kubernetes.io/projected/0159af74-ba53-40e0-9c67-aeb62d2e90de-kube-api-access-p8vrw\") pod \"authorino-8988bbd85-dcst4\" (UID: \"0159af74-ba53-40e0-9c67-aeb62d2e90de\") " pod="kuadrant-system/authorino-8988bbd85-dcst4" Jul 17 01:21:25.559531 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.559497 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8vrw\" (UniqueName: \"kubernetes.io/projected/0159af74-ba53-40e0-9c67-aeb62d2e90de-kube-api-access-p8vrw\") pod \"authorino-8988bbd85-dcst4\" (UID: \"0159af74-ba53-40e0-9c67-aeb62d2e90de\") " pod="kuadrant-system/authorino-8988bbd85-dcst4" Jul 17 01:21:25.583363 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.583334 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8988bbd85-dcst4" Jul 17 01:21:25.583776 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.583334 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-phv97" Jul 17 01:21:25.591055 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.591032 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8988bbd85-dcst4" Jul 17 01:21:25.593989 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.593970 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-phv97" Jul 17 01:21:25.652849 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.652822 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8vrw\" (UniqueName: \"kubernetes.io/projected/0159af74-ba53-40e0-9c67-aeb62d2e90de-kube-api-access-p8vrw\") pod \"0159af74-ba53-40e0-9c67-aeb62d2e90de\" (UID: \"0159af74-ba53-40e0-9c67-aeb62d2e90de\") " Jul 17 01:21:25.652987 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.652931 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cj9b\" (UniqueName: \"kubernetes.io/projected/9d02cba3-4a3d-4262-9c1b-a85277d19ae4-kube-api-access-4cj9b\") pod \"9d02cba3-4a3d-4262-9c1b-a85277d19ae4\" (UID: \"9d02cba3-4a3d-4262-9c1b-a85277d19ae4\") " Jul 17 01:21:25.653041 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.653011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmpzs\" (UniqueName: \"kubernetes.io/projected/5493e9ef-22ea-4ced-8802-73bdd1dd6958-kube-api-access-cmpzs\") pod \"authorino-5bb5679577-4rmnf\" (UID: \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\") " pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:21:25.653107 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.653092 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5493e9ef-22ea-4ced-8802-73bdd1dd6958-tls-cert\") pod \"authorino-5bb5679577-4rmnf\" (UID: \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\") " pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:21:25.655047 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.655013 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0159af74-ba53-40e0-9c67-aeb62d2e90de-kube-api-access-p8vrw" (OuterVolumeSpecName: "kube-api-access-p8vrw") pod "0159af74-ba53-40e0-9c67-aeb62d2e90de" (UID: "0159af74-ba53-40e0-9c67-aeb62d2e90de"). InnerVolumeSpecName "kube-api-access-p8vrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:21:25.655188 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.655043 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d02cba3-4a3d-4262-9c1b-a85277d19ae4-kube-api-access-4cj9b" (OuterVolumeSpecName: "kube-api-access-4cj9b") pod "9d02cba3-4a3d-4262-9c1b-a85277d19ae4" (UID: "9d02cba3-4a3d-4262-9c1b-a85277d19ae4"). InnerVolumeSpecName "kube-api-access-4cj9b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:21:25.753784 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.753744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmpzs\" (UniqueName: \"kubernetes.io/projected/5493e9ef-22ea-4ced-8802-73bdd1dd6958-kube-api-access-cmpzs\") pod \"authorino-5bb5679577-4rmnf\" (UID: \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\") " pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:21:25.753961 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.753817 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5493e9ef-22ea-4ced-8802-73bdd1dd6958-tls-cert\") pod \"authorino-5bb5679577-4rmnf\" (UID: \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\") " pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:21:25.753961 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.753849 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cj9b\" (UniqueName: \"kubernetes.io/projected/9d02cba3-4a3d-4262-9c1b-a85277d19ae4-kube-api-access-4cj9b\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:21:25.753961 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.753859 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8vrw\" (UniqueName: \"kubernetes.io/projected/0159af74-ba53-40e0-9c67-aeb62d2e90de-kube-api-access-p8vrw\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:21:25.756437 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.756406 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5493e9ef-22ea-4ced-8802-73bdd1dd6958-tls-cert\") pod \"authorino-5bb5679577-4rmnf\" (UID: \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\") " pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:21:25.761482 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.761457 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmpzs\" (UniqueName: \"kubernetes.io/projected/5493e9ef-22ea-4ced-8802-73bdd1dd6958-kube-api-access-cmpzs\") pod \"authorino-5bb5679577-4rmnf\" (UID: \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\") " pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:21:25.825035 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.825001 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:21:25.960070 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:25.960041 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bb5679577-4rmnf"] Jul 17 01:21:25.961185 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:21:25.961141 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5493e9ef_22ea_4ced_8802_73bdd1dd6958.slice/crio-3ab1575b326df8c1c29953716aa118552f287551076859f39d6dbe71b761f79a WatchSource:0}: Error finding container 3ab1575b326df8c1c29953716aa118552f287551076859f39d6dbe71b761f79a: Status 404 returned error can't find the container with id 3ab1575b326df8c1c29953716aa118552f287551076859f39d6dbe71b761f79a Jul 17 01:21:26.590805 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.590702 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bb5679577-4rmnf" event={"ID":"5493e9ef-22ea-4ced-8802-73bdd1dd6958","Type":"ContainerStarted","Data":"f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88"} Jul 17 01:21:26.590805 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.590753 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bb5679577-4rmnf" event={"ID":"5493e9ef-22ea-4ced-8802-73bdd1dd6958","Type":"ContainerStarted","Data":"3ab1575b326df8c1c29953716aa118552f287551076859f39d6dbe71b761f79a"} Jul 17 01:21:26.591314 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.590837 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8988bbd85-dcst4" Jul 17 01:21:26.591314 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.590996 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-phv97" Jul 17 01:21:26.607127 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.607061 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5bb5679577-4rmnf" podStartSLOduration=1.3095305879999999 podStartE2EDuration="1.607044204s" totalImagesPullingTime="297.513616ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:21:26.604810341 +0000 UTC m=+452.982734749" watchObservedRunningTime="2026-07-17 01:21:26.607044204 +0000 UTC m=+452.984968611" Jul 17 01:21:26.627919 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.627887 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phv97"] Jul 17 01:21:26.634509 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.634484 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-phv97"] Jul 17 01:21:26.637891 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.637868 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wdjsq"] Jul 17 01:21:26.638105 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.638078 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-wdjsq" podUID="d5c10ef9-83a9-4fc3-999e-c5327cc29b8e" containerName="authorino" containerID="cri-o://55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8" gracePeriod=30 Jul 17 01:21:26.657406 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.657381 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8988bbd85-dcst4"] Jul 17 01:21:26.659639 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.659617 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8988bbd85-dcst4"] Jul 17 01:21:26.887695 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.887661 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wdjsq" Jul 17 01:21:26.963356 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.963318 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kct9h\" (UniqueName: \"kubernetes.io/projected/d5c10ef9-83a9-4fc3-999e-c5327cc29b8e-kube-api-access-kct9h\") pod \"d5c10ef9-83a9-4fc3-999e-c5327cc29b8e\" (UID: \"d5c10ef9-83a9-4fc3-999e-c5327cc29b8e\") " Jul 17 01:21:26.965547 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:26.965522 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5c10ef9-83a9-4fc3-999e-c5327cc29b8e-kube-api-access-kct9h" (OuterVolumeSpecName: "kube-api-access-kct9h") pod "d5c10ef9-83a9-4fc3-999e-c5327cc29b8e" (UID: "d5c10ef9-83a9-4fc3-999e-c5327cc29b8e"). InnerVolumeSpecName "kube-api-access-kct9h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:21:27.064847 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.064802 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kct9h\" (UniqueName: \"kubernetes.io/projected/d5c10ef9-83a9-4fc3-999e-c5327cc29b8e-kube-api-access-kct9h\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:21:27.597590 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.597557 2564 generic.go:356] "Generic (PLEG): container finished" podID="d5c10ef9-83a9-4fc3-999e-c5327cc29b8e" containerID="55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8" exitCode=0 Jul 17 01:21:27.598062 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.597601 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-wdjsq" Jul 17 01:21:27.598062 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.597652 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wdjsq" event={"ID":"d5c10ef9-83a9-4fc3-999e-c5327cc29b8e","Type":"ContainerDied","Data":"55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8"} Jul 17 01:21:27.598062 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.597700 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-wdjsq" event={"ID":"d5c10ef9-83a9-4fc3-999e-c5327cc29b8e","Type":"ContainerDied","Data":"e4bde07d349cfc13e7a629e217e4bea110116b7fed427c7c3e7bff542a68a3f7"} Jul 17 01:21:27.598062 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.597726 2564 scope.go:117] "RemoveContainer" containerID="55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8" Jul 17 01:21:27.606817 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.606801 2564 scope.go:117] "RemoveContainer" containerID="55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8" Jul 17 01:21:27.607098 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:21:27.607065 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8\": container with ID starting with 55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8 not found: ID does not exist" containerID="55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8" Jul 17 01:21:27.607143 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.607106 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8"} err="failed to get container status \"55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8\": rpc error: code = NotFound desc = could not find container \"55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8\": container with ID starting with 55fbe25275cd2bed182e7fb154ff1c6909416a1976fd60e4e7041b53c36daaf8 not found: ID does not exist" Jul 17 01:21:27.619037 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.619012 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wdjsq"] Jul 17 01:21:27.622831 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:27.622812 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-wdjsq"] Jul 17 01:21:28.133921 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:28.133889 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0159af74-ba53-40e0-9c67-aeb62d2e90de" path="/var/lib/kubelet/pods/0159af74-ba53-40e0-9c67-aeb62d2e90de/volumes" Jul 17 01:21:28.134121 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:28.134109 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d02cba3-4a3d-4262-9c1b-a85277d19ae4" path="/var/lib/kubelet/pods/9d02cba3-4a3d-4262-9c1b-a85277d19ae4/volumes" Jul 17 01:21:28.134323 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:28.134311 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5c10ef9-83a9-4fc3-999e-c5327cc29b8e" path="/var/lib/kubelet/pods/d5c10ef9-83a9-4fc3-999e-c5327cc29b8e/volumes" Jul 17 01:21:31.247991 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.247954 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75646995f5-rxqgl"] Jul 17 01:21:31.248391 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.248377 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5c10ef9-83a9-4fc3-999e-c5327cc29b8e" containerName="authorino" Jul 17 01:21:31.248391 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.248393 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5c10ef9-83a9-4fc3-999e-c5327cc29b8e" containerName="authorino" Jul 17 01:21:31.248476 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.248459 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5c10ef9-83a9-4fc3-999e-c5327cc29b8e" containerName="authorino" Jul 17 01:21:31.256176 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.256139 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:31.259795 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.259766 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75646995f5-rxqgl"] Jul 17 01:21:31.261337 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.261321 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 17 01:21:31.262043 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.262028 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 17 01:21:31.262494 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.262478 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w742h\"" Jul 17 01:21:31.404815 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.404779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f710bf92-2748-465d-99f9-076350ab3efe-webhook-cert\") pod \"maas-controller-75646995f5-rxqgl\" (UID: \"f710bf92-2748-465d-99f9-076350ab3efe\") " pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:31.405011 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.404853 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5pd5\" (UniqueName: \"kubernetes.io/projected/f710bf92-2748-465d-99f9-076350ab3efe-kube-api-access-x5pd5\") pod \"maas-controller-75646995f5-rxqgl\" (UID: \"f710bf92-2748-465d-99f9-076350ab3efe\") " pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:31.414072 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.414038 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f988b7f6f-zbtwh"] Jul 17 01:21:31.417561 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.417540 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:31.426274 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.426245 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-zbtwh"] Jul 17 01:21:31.505762 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.505671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5pd5\" (UniqueName: \"kubernetes.io/projected/f710bf92-2748-465d-99f9-076350ab3efe-kube-api-access-x5pd5\") pod \"maas-controller-75646995f5-rxqgl\" (UID: \"f710bf92-2748-465d-99f9-076350ab3efe\") " pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:31.505762 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.505736 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f24r2\" (UniqueName: \"kubernetes.io/projected/a0f2cc71-54a6-48eb-af19-78d380f612b6-kube-api-access-f24r2\") pod \"maas-controller-f988b7f6f-zbtwh\" (UID: \"a0f2cc71-54a6-48eb-af19-78d380f612b6\") " pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:31.505762 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.505763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f710bf92-2748-465d-99f9-076350ab3efe-webhook-cert\") pod \"maas-controller-75646995f5-rxqgl\" (UID: \"f710bf92-2748-465d-99f9-076350ab3efe\") " pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:31.506046 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.505781 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f2cc71-54a6-48eb-af19-78d380f612b6-webhook-cert\") pod \"maas-controller-f988b7f6f-zbtwh\" (UID: \"a0f2cc71-54a6-48eb-af19-78d380f612b6\") " pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:31.508418 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.508389 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f710bf92-2748-465d-99f9-076350ab3efe-webhook-cert\") pod \"maas-controller-75646995f5-rxqgl\" (UID: \"f710bf92-2748-465d-99f9-076350ab3efe\") " pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:31.513602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.513578 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5pd5\" (UniqueName: \"kubernetes.io/projected/f710bf92-2748-465d-99f9-076350ab3efe-kube-api-access-x5pd5\") pod \"maas-controller-75646995f5-rxqgl\" (UID: \"f710bf92-2748-465d-99f9-076350ab3efe\") " pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:31.567436 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.567392 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:31.606877 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.606827 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f24r2\" (UniqueName: \"kubernetes.io/projected/a0f2cc71-54a6-48eb-af19-78d380f612b6-kube-api-access-f24r2\") pod \"maas-controller-f988b7f6f-zbtwh\" (UID: \"a0f2cc71-54a6-48eb-af19-78d380f612b6\") " pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:31.607074 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.606904 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f2cc71-54a6-48eb-af19-78d380f612b6-webhook-cert\") pod \"maas-controller-f988b7f6f-zbtwh\" (UID: \"a0f2cc71-54a6-48eb-af19-78d380f612b6\") " pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:31.609648 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.609618 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f2cc71-54a6-48eb-af19-78d380f612b6-webhook-cert\") pod \"maas-controller-f988b7f6f-zbtwh\" (UID: \"a0f2cc71-54a6-48eb-af19-78d380f612b6\") " pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:31.614823 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.614799 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f24r2\" (UniqueName: \"kubernetes.io/projected/a0f2cc71-54a6-48eb-af19-78d380f612b6-kube-api-access-f24r2\") pod \"maas-controller-f988b7f6f-zbtwh\" (UID: \"a0f2cc71-54a6-48eb-af19-78d380f612b6\") " pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:31.707041 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.707016 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75646995f5-rxqgl"] Jul 17 01:21:31.708732 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:21:31.708700 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf710bf92_2748_465d_99f9_076350ab3efe.slice/crio-7e873b88de7a7f80b942904f36eefcaa473f48018f28d47386095ea672276782 WatchSource:0}: Error finding container 7e873b88de7a7f80b942904f36eefcaa473f48018f28d47386095ea672276782: Status 404 returned error can't find the container with id 7e873b88de7a7f80b942904f36eefcaa473f48018f28d47386095ea672276782 Jul 17 01:21:31.728283 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.728254 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:31.860877 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:31.860850 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-zbtwh"] Jul 17 01:21:31.862599 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:21:31.862575 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f2cc71_54a6_48eb_af19_78d380f612b6.slice/crio-b147dad9fc722f13800475bce5d481da766d90b4d9399b52d530da45b57b5cdc WatchSource:0}: Error finding container b147dad9fc722f13800475bce5d481da766d90b4d9399b52d530da45b57b5cdc: Status 404 returned error can't find the container with id b147dad9fc722f13800475bce5d481da766d90b4d9399b52d530da45b57b5cdc Jul 17 01:21:32.630876 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:32.630838 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" event={"ID":"a0f2cc71-54a6-48eb-af19-78d380f612b6","Type":"ContainerStarted","Data":"b147dad9fc722f13800475bce5d481da766d90b4d9399b52d530da45b57b5cdc"} Jul 17 01:21:32.633604 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:32.633576 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-rxqgl" event={"ID":"f710bf92-2748-465d-99f9-076350ab3efe","Type":"ContainerStarted","Data":"7e873b88de7a7f80b942904f36eefcaa473f48018f28d47386095ea672276782"} Jul 17 01:21:34.649083 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:34.649044 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" event={"ID":"a0f2cc71-54a6-48eb-af19-78d380f612b6","Type":"ContainerStarted","Data":"9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5"} Jul 17 01:21:34.650745 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:34.650720 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-rxqgl" event={"ID":"f710bf92-2748-465d-99f9-076350ab3efe","Type":"ContainerStarted","Data":"9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a"} Jul 17 01:21:35.658311 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:35.658280 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:35.676194 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:35.676118 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75646995f5-rxqgl" podStartSLOduration=1.858720326 podStartE2EDuration="4.676102612s" totalImagesPullingTime="2.817382286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:21:35.675461826 +0000 UTC m=+462.053386234" watchObservedRunningTime="2026-07-17 01:21:35.676102612 +0000 UTC m=+462.054027019" Jul 17 01:21:35.693409 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:35.693353 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" podStartSLOduration=2.041968298 podStartE2EDuration="4.693337094s" totalImagesPullingTime="2.651368796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:21:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:21:35.690717678 +0000 UTC m=+462.068642077" watchObservedRunningTime="2026-07-17 01:21:35.693337094 +0000 UTC m=+462.071261501" Jul 17 01:21:36.663183 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:36.663122 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:46.665496 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:46.665466 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:47.670699 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:47.670670 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:21:47.715477 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:47.715437 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75646995f5-rxqgl"] Jul 17 01:21:47.715694 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:47.715639 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75646995f5-rxqgl" podUID="f710bf92-2748-465d-99f9-076350ab3efe" containerName="manager" containerID="cri-o://9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a" gracePeriod=30 Jul 17 01:21:48.518677 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.518642 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:48.664090 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.664053 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f710bf92-2748-465d-99f9-076350ab3efe-webhook-cert\") pod \"f710bf92-2748-465d-99f9-076350ab3efe\" (UID: \"f710bf92-2748-465d-99f9-076350ab3efe\") " Jul 17 01:21:48.664325 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.664153 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5pd5\" (UniqueName: \"kubernetes.io/projected/f710bf92-2748-465d-99f9-076350ab3efe-kube-api-access-x5pd5\") pod \"f710bf92-2748-465d-99f9-076350ab3efe\" (UID: \"f710bf92-2748-465d-99f9-076350ab3efe\") " Jul 17 01:21:48.666516 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.666480 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f710bf92-2748-465d-99f9-076350ab3efe-kube-api-access-x5pd5" (OuterVolumeSpecName: "kube-api-access-x5pd5") pod "f710bf92-2748-465d-99f9-076350ab3efe" (UID: "f710bf92-2748-465d-99f9-076350ab3efe"). InnerVolumeSpecName "kube-api-access-x5pd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:21:48.666516 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.666489 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f710bf92-2748-465d-99f9-076350ab3efe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f710bf92-2748-465d-99f9-076350ab3efe" (UID: "f710bf92-2748-465d-99f9-076350ab3efe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:21:48.739416 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.739381 2564 generic.go:356] "Generic (PLEG): container finished" podID="f710bf92-2748-465d-99f9-076350ab3efe" containerID="9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a" exitCode=0 Jul 17 01:21:48.739818 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.739426 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75646995f5-rxqgl" Jul 17 01:21:48.739818 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.739475 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-rxqgl" event={"ID":"f710bf92-2748-465d-99f9-076350ab3efe","Type":"ContainerDied","Data":"9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a"} Jul 17 01:21:48.739818 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.739528 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75646995f5-rxqgl" event={"ID":"f710bf92-2748-465d-99f9-076350ab3efe","Type":"ContainerDied","Data":"7e873b88de7a7f80b942904f36eefcaa473f48018f28d47386095ea672276782"} Jul 17 01:21:48.739818 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.739553 2564 scope.go:117] "RemoveContainer" containerID="9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a" Jul 17 01:21:48.748645 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.748629 2564 scope.go:117] "RemoveContainer" containerID="9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a" Jul 17 01:21:48.748908 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:21:48.748888 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a\": container with ID starting with 9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a not found: ID does not exist" containerID="9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a" Jul 17 01:21:48.748957 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.748918 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a"} err="failed to get container status \"9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a\": rpc error: code = NotFound desc = could not find container \"9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a\": container with ID starting with 9a21b44bd3315f9535fa94b89c03c3cfbcc8743ddf33cdbc88df8ff403487e0a not found: ID does not exist" Jul 17 01:21:48.762921 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.762890 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75646995f5-rxqgl"] Jul 17 01:21:48.764926 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.764906 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f710bf92-2748-465d-99f9-076350ab3efe-webhook-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:21:48.764983 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.764934 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5pd5\" (UniqueName: \"kubernetes.io/projected/f710bf92-2748-465d-99f9-076350ab3efe-kube-api-access-x5pd5\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:21:48.765279 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:48.765264 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75646995f5-rxqgl"] Jul 17 01:21:50.133743 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:21:50.133714 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f710bf92-2748-465d-99f9-076350ab3efe" path="/var/lib/kubelet/pods/f710bf92-2748-465d-99f9-076350ab3efe/volumes" Jul 17 01:22:07.336304 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.336266 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps"] Jul 17 01:22:07.336733 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.336716 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f710bf92-2748-465d-99f9-076350ab3efe" containerName="manager" Jul 17 01:22:07.336778 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.336736 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f710bf92-2748-465d-99f9-076350ab3efe" containerName="manager" Jul 17 01:22:07.336830 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.336820 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f710bf92-2748-465d-99f9-076350ab3efe" containerName="manager" Jul 17 01:22:07.341611 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.341590 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.344117 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.344095 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 17 01:22:07.351831 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.351802 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps"] Jul 17 01:22:07.424362 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.424321 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140b6f2e-fec4-4fd0-ad32-10bf54a86d6b-openshift-service-ca\") pod \"maas-api-7954f7d7ff-rspps\" (UID: \"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.424362 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.424366 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqhtn\" (UniqueName: \"kubernetes.io/projected/140b6f2e-fec4-4fd0-ad32-10bf54a86d6b-kube-api-access-cqhtn\") pod \"maas-api-7954f7d7ff-rspps\" (UID: \"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.424597 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.424483 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140b6f2e-fec4-4fd0-ad32-10bf54a86d6b-maas-api-tls\") pod \"maas-api-7954f7d7ff-rspps\" (UID: \"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.525879 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.525837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140b6f2e-fec4-4fd0-ad32-10bf54a86d6b-maas-api-tls\") pod \"maas-api-7954f7d7ff-rspps\" (UID: \"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.525879 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.525888 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140b6f2e-fec4-4fd0-ad32-10bf54a86d6b-openshift-service-ca\") pod \"maas-api-7954f7d7ff-rspps\" (UID: \"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.526106 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.526005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqhtn\" (UniqueName: \"kubernetes.io/projected/140b6f2e-fec4-4fd0-ad32-10bf54a86d6b-kube-api-access-cqhtn\") pod \"maas-api-7954f7d7ff-rspps\" (UID: \"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.526516 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.526499 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/140b6f2e-fec4-4fd0-ad32-10bf54a86d6b-openshift-service-ca\") pod \"maas-api-7954f7d7ff-rspps\" (UID: \"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.528612 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.528594 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/140b6f2e-fec4-4fd0-ad32-10bf54a86d6b-maas-api-tls\") pod \"maas-api-7954f7d7ff-rspps\" (UID: \"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.535756 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.535728 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqhtn\" (UniqueName: \"kubernetes.io/projected/140b6f2e-fec4-4fd0-ad32-10bf54a86d6b-kube-api-access-cqhtn\") pod \"maas-api-7954f7d7ff-rspps\" (UID: \"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b\") " pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.653212 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.653177 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:07.800897 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.800872 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps"] Jul 17 01:22:07.801868 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:22:07.801845 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod140b6f2e_fec4_4fd0_ad32_10bf54a86d6b.slice/crio-0ade1b5877773d2b34ca9df5f16c441aeb6b970b486c770bfbb8f345f77c35cc WatchSource:0}: Error finding container 0ade1b5877773d2b34ca9df5f16c441aeb6b970b486c770bfbb8f345f77c35cc: Status 404 returned error can't find the container with id 0ade1b5877773d2b34ca9df5f16c441aeb6b970b486c770bfbb8f345f77c35cc Jul 17 01:22:07.842051 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.842013 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" event={"ID":"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b","Type":"ContainerStarted","Data":"0ade1b5877773d2b34ca9df5f16c441aeb6b970b486c770bfbb8f345f77c35cc"} Jul 17 01:22:07.848940 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.848905 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-pr7c6"] Jul 17 01:22:07.852366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.852345 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:07.854940 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.854922 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 17 01:22:07.864883 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.864858 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-pr7c6"] Jul 17 01:22:07.886995 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.886963 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-j5v7l"] Jul 17 01:22:07.890369 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.890347 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:07.893047 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.893029 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rzdh2\"" Jul 17 01:22:07.903103 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.903077 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-j5v7l"] Jul 17 01:22:07.930053 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.929943 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mbvp\" (UniqueName: \"kubernetes.io/projected/d8be559c-ba5a-4a21-9baf-ea63b397dab6-kube-api-access-7mbvp\") pod \"payload-processing-5bf47cd988-j5v7l\" (UID: \"d8be559c-ba5a-4a21-9baf-ea63b397dab6\") " pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:07.930053 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.930009 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b568047d-f58b-48d2-b7c8-f5276c47cbd6-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-pr7c6\" (UID: \"b568047d-f58b-48d2-b7c8-f5276c47cbd6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:07.930391 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.930057 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8be559c-ba5a-4a21-9baf-ea63b397dab6-plugins-config-volume\") pod \"payload-processing-5bf47cd988-j5v7l\" (UID: \"d8be559c-ba5a-4a21-9baf-ea63b397dab6\") " pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:07.930391 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:07.930147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7mdm\" (UniqueName: \"kubernetes.io/projected/b568047d-f58b-48d2-b7c8-f5276c47cbd6-kube-api-access-p7mdm\") pod \"payload-pre-processing-7677b774bb-pr7c6\" (UID: \"b568047d-f58b-48d2-b7c8-f5276c47cbd6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:08.031193 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.031135 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7mdm\" (UniqueName: \"kubernetes.io/projected/b568047d-f58b-48d2-b7c8-f5276c47cbd6-kube-api-access-p7mdm\") pod \"payload-pre-processing-7677b774bb-pr7c6\" (UID: \"b568047d-f58b-48d2-b7c8-f5276c47cbd6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:08.031359 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.031271 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mbvp\" (UniqueName: \"kubernetes.io/projected/d8be559c-ba5a-4a21-9baf-ea63b397dab6-kube-api-access-7mbvp\") pod \"payload-processing-5bf47cd988-j5v7l\" (UID: \"d8be559c-ba5a-4a21-9baf-ea63b397dab6\") " pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:08.031359 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.031323 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b568047d-f58b-48d2-b7c8-f5276c47cbd6-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-pr7c6\" (UID: \"b568047d-f58b-48d2-b7c8-f5276c47cbd6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:08.031490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.031362 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8be559c-ba5a-4a21-9baf-ea63b397dab6-plugins-config-volume\") pod \"payload-processing-5bf47cd988-j5v7l\" (UID: \"d8be559c-ba5a-4a21-9baf-ea63b397dab6\") " pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:08.032106 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.032077 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b568047d-f58b-48d2-b7c8-f5276c47cbd6-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-pr7c6\" (UID: \"b568047d-f58b-48d2-b7c8-f5276c47cbd6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:08.032306 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.032283 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d8be559c-ba5a-4a21-9baf-ea63b397dab6-plugins-config-volume\") pod \"payload-processing-5bf47cd988-j5v7l\" (UID: \"d8be559c-ba5a-4a21-9baf-ea63b397dab6\") " pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:08.040127 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.040099 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7mdm\" (UniqueName: \"kubernetes.io/projected/b568047d-f58b-48d2-b7c8-f5276c47cbd6-kube-api-access-p7mdm\") pod \"payload-pre-processing-7677b774bb-pr7c6\" (UID: \"b568047d-f58b-48d2-b7c8-f5276c47cbd6\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:08.040291 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.040269 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mbvp\" (UniqueName: \"kubernetes.io/projected/d8be559c-ba5a-4a21-9baf-ea63b397dab6-kube-api-access-7mbvp\") pod \"payload-processing-5bf47cd988-j5v7l\" (UID: \"d8be559c-ba5a-4a21-9baf-ea63b397dab6\") " pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:08.163328 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.163292 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:08.201814 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.201717 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:08.327331 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.327303 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-pr7c6"] Jul 17 01:22:08.328099 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:22:08.328074 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb568047d_f58b_48d2_b7c8_f5276c47cbd6.slice/crio-5d52b8cf011090a3856f1923fa39cd8f65f58668a55e5f55ec70790bc45b7488 WatchSource:0}: Error finding container 5d52b8cf011090a3856f1923fa39cd8f65f58668a55e5f55ec70790bc45b7488: Status 404 returned error can't find the container with id 5d52b8cf011090a3856f1923fa39cd8f65f58668a55e5f55ec70790bc45b7488 Jul 17 01:22:08.367017 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:22:08.363706 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8be559c_ba5a_4a21_9baf_ea63b397dab6.slice/crio-67ccdd6b269066b44ab1355f960ecacdbfed68c92b78da79af06549514ca56d5 WatchSource:0}: Error finding container 67ccdd6b269066b44ab1355f960ecacdbfed68c92b78da79af06549514ca56d5: Status 404 returned error can't find the container with id 67ccdd6b269066b44ab1355f960ecacdbfed68c92b78da79af06549514ca56d5 Jul 17 01:22:08.367017 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.364054 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-j5v7l"] Jul 17 01:22:08.852619 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.852265 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" event={"ID":"b568047d-f58b-48d2-b7c8-f5276c47cbd6","Type":"ContainerStarted","Data":"5d52b8cf011090a3856f1923fa39cd8f65f58668a55e5f55ec70790bc45b7488"} Jul 17 01:22:08.854208 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:08.854144 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" event={"ID":"d8be559c-ba5a-4a21-9baf-ea63b397dab6","Type":"ContainerStarted","Data":"67ccdd6b269066b44ab1355f960ecacdbfed68c92b78da79af06549514ca56d5"} Jul 17 01:22:10.869459 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:10.869424 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" event={"ID":"140b6f2e-fec4-4fd0-ad32-10bf54a86d6b","Type":"ContainerStarted","Data":"f8daf462347bf05f15a8c432fd5e66ee84ef79e590343bf00cc1e76dda5afee7"} Jul 17 01:22:10.869828 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:10.869530 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:10.885469 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:10.885292 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" podStartSLOduration=1.525786015 podStartE2EDuration="3.885273454s" totalImagesPullingTime="2.359487439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:22:10.884430861 +0000 UTC m=+497.262355268" watchObservedRunningTime="2026-07-17 01:22:10.885273454 +0000 UTC m=+497.263197864" Jul 17 01:22:11.876582 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:11.876538 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" event={"ID":"b568047d-f58b-48d2-b7c8-f5276c47cbd6","Type":"ContainerStarted","Data":"f691bb0f75b80832b56b43c817120459b6e639b6cd5bcc8ba72731482c5489de"} Jul 17 01:22:11.877031 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:11.876604 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:11.878058 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:11.878038 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" event={"ID":"d8be559c-ba5a-4a21-9baf-ea63b397dab6","Type":"ContainerStarted","Data":"769a64cc7918e2a7d25c407f8e2fc3ac984424991509a70ed25b39ae3bf3ef5b"} Jul 17 01:22:11.878183 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:11.878151 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:11.908088 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:11.908014 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" podStartSLOduration=2.3442808 podStartE2EDuration="4.907994253s" totalImagesPullingTime="2.563713453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:22:11.906486877 +0000 UTC m=+498.284411286" watchObservedRunningTime="2026-07-17 01:22:11.907994253 +0000 UTC m=+498.285918662" Jul 17 01:22:11.923263 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:11.923206 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" podStartSLOduration=2.390349583 podStartE2EDuration="4.923185498s" totalImagesPullingTime="2.532835915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:22:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:22:11.920538977 +0000 UTC m=+498.298463385" watchObservedRunningTime="2026-07-17 01:22:11.923185498 +0000 UTC m=+498.301109906" Jul 17 01:22:15.904275 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:15.904244 2564 generic.go:356] "Generic (PLEG): container finished" podID="d4ac076b-8b5c-4dc1-82db-6fa21f323ecf" containerID="6be70de2d016b5d5eb30d581be5e3a2f4afb5a438edd1e11f65e1e159eb30dbd" exitCode=0 Jul 17 01:22:15.904671 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:15.904289 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" event={"ID":"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf","Type":"ContainerDied","Data":"6be70de2d016b5d5eb30d581be5e3a2f4afb5a438edd1e11f65e1e159eb30dbd"} Jul 17 01:22:15.904723 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:15.904682 2564 scope.go:117] "RemoveContainer" containerID="6be70de2d016b5d5eb30d581be5e3a2f4afb5a438edd1e11f65e1e159eb30dbd" Jul 17 01:22:15.905681 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:15.905643 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:22:15.905780 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:15.905713 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:22:15.905780 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:15.905749 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:22:16.883451 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:16.883419 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7954f7d7ff-rspps" Jul 17 01:22:16.914850 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:16.914805 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" event={"ID":"d4ac076b-8b5c-4dc1-82db-6fa21f323ecf","Type":"ContainerStarted","Data":"38c2336fdff7dbad9023918077684bd6948c211f6e85183f242d41dc1a4236fb"} Jul 17 01:22:17.098910 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:17.098872 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:22:17.100063 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:17.100044 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-dxtsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 17 01:22:17.100119 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:17.100087 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" podUID="d4ac076b-8b5c-4dc1-82db-6fa21f323ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 17 01:22:18.099238 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:18.099200 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-dxtsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 17 01:22:18.099730 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:18.099260 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" podUID="d4ac076b-8b5c-4dc1-82db-6fa21f323ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 17 01:22:19.098502 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:19.098457 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-dxtsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 17 01:22:19.098713 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:19.098522 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" podUID="d4ac076b-8b5c-4dc1-82db-6fa21f323ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 17 01:22:20.098509 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:20.098468 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-dxtsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 17 01:22:20.098909 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:20.098528 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" podUID="d4ac076b-8b5c-4dc1-82db-6fa21f323ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 17 01:22:20.347586 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:20.347544 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:22:21.103491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:21.103459 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:22:21.104467 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:21.104445 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-dxtsn" Jul 17 01:22:22.886365 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:22.886337 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-pr7c6" Jul 17 01:22:22.886757 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:22.886549 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-j5v7l" Jul 17 01:22:28.773906 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.773873 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs"] Jul 17 01:22:28.777099 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.777081 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.780712 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.780691 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 17 01:22:28.780853 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.780694 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-bjgtp\"" Jul 17 01:22:28.780973 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.780954 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 17 01:22:28.781200 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.781185 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 17 01:22:28.785710 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.785688 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs"] Jul 17 01:22:28.830790 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.830755 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.830983 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.830823 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knn7r\" (UniqueName: \"kubernetes.io/projected/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-kube-api-access-knn7r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.830983 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.830885 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.830983 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.830911 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.831115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.831016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.831115 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.831087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.932240 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.932204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.932427 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.932258 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.932427 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.932305 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knn7r\" (UniqueName: \"kubernetes.io/projected/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-kube-api-access-knn7r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.932547 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.932446 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.932547 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.932481 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.932547 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.932530 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.932690 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.932614 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.932690 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.932644 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.932877 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.932859 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.934751 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.934727 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.934973 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.934957 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:28.939751 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:28.939727 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knn7r\" (UniqueName: \"kubernetes.io/projected/b7b53d29-4b24-4f16-90a8-3cc35bacfc0c-kube-api-access-knn7r\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs\" (UID: \"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:29.093415 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:29.093324 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:29.235202 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:29.235117 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs"] Jul 17 01:22:29.237228 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:22:29.237197 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b53d29_4b24_4f16_90a8_3cc35bacfc0c.slice/crio-28f7f7698fa6665635e373cb78fbb5578b33709a6e4a5a7e9436cf9aae9f865b WatchSource:0}: Error finding container 28f7f7698fa6665635e373cb78fbb5578b33709a6e4a5a7e9436cf9aae9f865b: Status 404 returned error can't find the container with id 28f7f7698fa6665635e373cb78fbb5578b33709a6e4a5a7e9436cf9aae9f865b Jul 17 01:22:29.994633 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:29.994596 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" event={"ID":"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c","Type":"ContainerStarted","Data":"28f7f7698fa6665635e373cb78fbb5578b33709a6e4a5a7e9436cf9aae9f865b"} Jul 17 01:22:31.009183 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:31.009121 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" event={"ID":"b7b53d29-4b24-4f16-90a8-3cc35bacfc0c","Type":"ContainerStarted","Data":"5a9e14cd4349ca64dc6b0b273fa636b60a368499f77503b720647646e338036d"} Jul 17 01:22:31.025916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:31.025864 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" podStartSLOduration=1.8881477100000001 podStartE2EDuration="3.025850042s" totalImagesPullingTime="1.137702332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:22:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:22:31.025455501 +0000 UTC m=+517.403379910" watchObservedRunningTime="2026-07-17 01:22:31.025850042 +0000 UTC m=+517.403774450" Jul 17 01:22:39.093869 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:39.093837 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:39.094372 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:39.093996 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:39.106793 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:39.106769 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:22:40.072023 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:22:40.071994 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs" Jul 17 01:23:02.260356 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.260321 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2"] Jul 17 01:23:02.267024 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.267003 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.269765 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.269739 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 17 01:23:02.279870 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.279846 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2"] Jul 17 01:23:02.328609 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.328571 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djr4d\" (UniqueName: \"kubernetes.io/projected/397482b6-dd04-4551-be34-0ed3f191ac05-kube-api-access-djr4d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.328810 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.328642 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.328810 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.328668 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.328810 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.328693 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.328810 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.328720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.328810 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.328805 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/397482b6-dd04-4551-be34-0ed3f191ac05-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.430157 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.430113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djr4d\" (UniqueName: \"kubernetes.io/projected/397482b6-dd04-4551-be34-0ed3f191ac05-kube-api-access-djr4d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.430394 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.430226 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.430394 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.430250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.430394 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.430276 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.430545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.430405 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.430545 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.430507 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/397482b6-dd04-4551-be34-0ed3f191ac05-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.430690 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.430666 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.430755 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.430734 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.430817 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.430791 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.432806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.432780 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/397482b6-dd04-4551-be34-0ed3f191ac05-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.433079 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.433062 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/397482b6-dd04-4551-be34-0ed3f191ac05-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.438396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.438370 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djr4d\" (UniqueName: \"kubernetes.io/projected/397482b6-dd04-4551-be34-0ed3f191ac05-kube-api-access-djr4d\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mcms2\" (UID: \"397482b6-dd04-4551-be34-0ed3f191ac05\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.577422 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.577328 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:02.720812 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:02.720765 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2"] Jul 17 01:23:02.722871 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:23:02.722837 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod397482b6_dd04_4551_be34_0ed3f191ac05.slice/crio-562c7914cfa873caa2aa7f911b196f8ba98e0d12d785c3014a4724ccabc91048 WatchSource:0}: Error finding container 562c7914cfa873caa2aa7f911b196f8ba98e0d12d785c3014a4724ccabc91048: Status 404 returned error can't find the container with id 562c7914cfa873caa2aa7f911b196f8ba98e0d12d785c3014a4724ccabc91048 Jul 17 01:23:03.192203 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:03.192143 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" event={"ID":"397482b6-dd04-4551-be34-0ed3f191ac05","Type":"ContainerStarted","Data":"cb6ed755ed534d548788c185e831069c4405e68a06334dad8f25b84805477864"} Jul 17 01:23:03.192382 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:03.192229 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" event={"ID":"397482b6-dd04-4551-be34-0ed3f191ac05","Type":"ContainerStarted","Data":"562c7914cfa873caa2aa7f911b196f8ba98e0d12d785c3014a4724ccabc91048"} Jul 17 01:23:03.211400 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:03.211338 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" podStartSLOduration=1.043021563 podStartE2EDuration="1.211317654s" totalImagesPullingTime="168.296091ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:23:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:23:03.210424699 +0000 UTC m=+549.588349149" watchObservedRunningTime="2026-07-17 01:23:03.211317654 +0000 UTC m=+549.589242063" Jul 17 01:23:12.578106 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:12.578068 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:12.578561 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:12.578145 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:12.590984 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:12.590955 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:13.263184 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:13.263139 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mcms2" Jul 17 01:23:33.161666 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.161629 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9"] Jul 17 01:23:33.165941 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.165906 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.168455 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.168435 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 17 01:23:33.174766 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.174742 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9"] Jul 17 01:23:33.210888 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.210860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.211035 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.210900 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.211035 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.210929 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.211035 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.211002 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.211142 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.211049 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.211142 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.211133 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5qdg\" (UniqueName: \"kubernetes.io/projected/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-kube-api-access-h5qdg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.311637 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.311596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.311853 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.311652 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.311853 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.311688 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.311853 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.311737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.312040 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.311862 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.312040 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.311958 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5qdg\" (UniqueName: \"kubernetes.io/projected/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-kube-api-access-h5qdg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.312145 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.312072 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.312145 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.312120 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.312374 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.312351 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.314298 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.314274 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.314454 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.314436 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.320693 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.320664 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5qdg\" (UniqueName: \"kubernetes.io/projected/8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2-kube-api-access-h5qdg\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-686v9\" (UID: \"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.480745 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.480652 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:33.620329 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:33.620297 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9"] Jul 17 01:23:33.621371 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:23:33.621345 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f5dfeda_b3ab_4322_83ce_1a7cf2ee10a2.slice/crio-16ca38cdb0affacf702e150d3e28d6ef30d3ab7727831690f7a7ee8489084476 WatchSource:0}: Error finding container 16ca38cdb0affacf702e150d3e28d6ef30d3ab7727831690f7a7ee8489084476: Status 404 returned error can't find the container with id 16ca38cdb0affacf702e150d3e28d6ef30d3ab7727831690f7a7ee8489084476 Jul 17 01:23:34.367736 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:34.367700 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" event={"ID":"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2","Type":"ContainerStarted","Data":"4a5d10ad2c88f74ab66c1fcd59f6ecbb1a2abfc256b7ce0a743f759333170d8f"} Jul 17 01:23:34.368125 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:34.367792 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" event={"ID":"8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2","Type":"ContainerStarted","Data":"16ca38cdb0affacf702e150d3e28d6ef30d3ab7727831690f7a7ee8489084476"} Jul 17 01:23:34.386982 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:34.386930 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" podStartSLOduration=1.209261678 podStartE2EDuration="1.386915887s" totalImagesPullingTime="177.654209ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:23:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:23:34.385517203 +0000 UTC m=+580.763441612" watchObservedRunningTime="2026-07-17 01:23:34.386915887 +0000 UTC m=+580.764840348" Jul 17 01:23:43.480844 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:43.480800 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:43.481364 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:43.480915 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:43.493719 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:43.493696 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:44.436355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:44.436327 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-686v9" Jul 17 01:23:49.266466 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.266408 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n"] Jul 17 01:23:49.270480 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.270456 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.272890 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.272867 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 17 01:23:49.282064 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.282039 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n"] Jul 17 01:23:49.347007 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.346975 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.347203 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.347014 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.347203 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.347039 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.347203 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.347086 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.347203 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.347117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd7c591-b985-404f-a61f-35d178fa81af-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.347203 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.347137 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dj4k\" (UniqueName: \"kubernetes.io/projected/4fd7c591-b985-404f-a61f-35d178fa81af-kube-api-access-2dj4k\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.448124 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.448086 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.448338 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.448134 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd7c591-b985-404f-a61f-35d178fa81af-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.448338 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.448181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dj4k\" (UniqueName: \"kubernetes.io/projected/4fd7c591-b985-404f-a61f-35d178fa81af-kube-api-access-2dj4k\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.448443 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.448363 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.448443 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.448401 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.448443 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.448437 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.448848 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.448822 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.449094 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.449071 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.449304 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.449218 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.450772 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.450741 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4fd7c591-b985-404f-a61f-35d178fa81af-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.451031 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.451014 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4fd7c591-b985-404f-a61f-35d178fa81af-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.456341 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.456317 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dj4k\" (UniqueName: \"kubernetes.io/projected/4fd7c591-b985-404f-a61f-35d178fa81af-kube-api-access-2dj4k\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n\" (UID: \"4fd7c591-b985-404f-a61f-35d178fa81af\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.581834 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.581757 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:49.721813 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:49.721781 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n"] Jul 17 01:23:49.723882 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:23:49.723860 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4fd7c591_b985_404f_a61f_35d178fa81af.slice/crio-03e45c845dba99e4a2ec4ad44044e64f23160dc8e56f489797d6e3a7263dddd0 WatchSource:0}: Error finding container 03e45c845dba99e4a2ec4ad44044e64f23160dc8e56f489797d6e3a7263dddd0: Status 404 returned error can't find the container with id 03e45c845dba99e4a2ec4ad44044e64f23160dc8e56f489797d6e3a7263dddd0 Jul 17 01:23:50.462590 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:50.462550 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" event={"ID":"4fd7c591-b985-404f-a61f-35d178fa81af","Type":"ContainerStarted","Data":"76e5f289052a9e59eac918aa9eaac5d36e1ea67dda101e47fabc966a7dda31ff"} Jul 17 01:23:50.463032 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:50.462600 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" event={"ID":"4fd7c591-b985-404f-a61f-35d178fa81af","Type":"ContainerStarted","Data":"03e45c845dba99e4a2ec4ad44044e64f23160dc8e56f489797d6e3a7263dddd0"} Jul 17 01:23:50.479338 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:50.479287 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" podStartSLOduration=1.282626877 podStartE2EDuration="1.479268415s" totalImagesPullingTime="196.641538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:23:50.477356814 +0000 UTC m=+596.855281223" watchObservedRunningTime="2026-07-17 01:23:50.479268415 +0000 UTC m=+596.857192823" Jul 17 01:23:54.046193 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:54.046151 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:23:54.048135 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:54.048118 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:23:59.581977 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:59.581941 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:59.582512 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:59.582008 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:23:59.594497 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:23:59.594470 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:24:00.529874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:00.529837 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n" Jul 17 01:24:36.675988 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.675942 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-685db4888-2swfm"] Jul 17 01:24:36.679439 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.679421 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685db4888-2swfm" Jul 17 01:24:36.688150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.687781 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-685db4888-2swfm"] Jul 17 01:24:36.802008 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.801969 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b071c93-8839-46d9-a60e-7a79decb8e10-tls-cert\") pod \"authorino-685db4888-2swfm\" (UID: \"0b071c93-8839-46d9-a60e-7a79decb8e10\") " pod="kuadrant-system/authorino-685db4888-2swfm" Jul 17 01:24:36.802207 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.802054 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74htk\" (UniqueName: \"kubernetes.io/projected/0b071c93-8839-46d9-a60e-7a79decb8e10-kube-api-access-74htk\") pod \"authorino-685db4888-2swfm\" (UID: \"0b071c93-8839-46d9-a60e-7a79decb8e10\") " pod="kuadrant-system/authorino-685db4888-2swfm" Jul 17 01:24:36.903427 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.903389 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b071c93-8839-46d9-a60e-7a79decb8e10-tls-cert\") pod \"authorino-685db4888-2swfm\" (UID: \"0b071c93-8839-46d9-a60e-7a79decb8e10\") " pod="kuadrant-system/authorino-685db4888-2swfm" Jul 17 01:24:36.903608 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.903476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74htk\" (UniqueName: \"kubernetes.io/projected/0b071c93-8839-46d9-a60e-7a79decb8e10-kube-api-access-74htk\") pod \"authorino-685db4888-2swfm\" (UID: \"0b071c93-8839-46d9-a60e-7a79decb8e10\") " pod="kuadrant-system/authorino-685db4888-2swfm" Jul 17 01:24:36.906156 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.906112 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0b071c93-8839-46d9-a60e-7a79decb8e10-tls-cert\") pod \"authorino-685db4888-2swfm\" (UID: \"0b071c93-8839-46d9-a60e-7a79decb8e10\") " pod="kuadrant-system/authorino-685db4888-2swfm" Jul 17 01:24:36.911793 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.911761 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74htk\" (UniqueName: \"kubernetes.io/projected/0b071c93-8839-46d9-a60e-7a79decb8e10-kube-api-access-74htk\") pod \"authorino-685db4888-2swfm\" (UID: \"0b071c93-8839-46d9-a60e-7a79decb8e10\") " pod="kuadrant-system/authorino-685db4888-2swfm" Jul 17 01:24:36.995074 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:36.994967 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-685db4888-2swfm" Jul 17 01:24:37.136441 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:37.136412 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-685db4888-2swfm"] Jul 17 01:24:37.137929 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:24:37.137894 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b071c93_8839_46d9_a60e_7a79decb8e10.slice/crio-895d32034fd933d89cc74745c82ca26f382370832c1ff467027ad75e6d46abbb WatchSource:0}: Error finding container 895d32034fd933d89cc74745c82ca26f382370832c1ff467027ad75e6d46abbb: Status 404 returned error can't find the container with id 895d32034fd933d89cc74745c82ca26f382370832c1ff467027ad75e6d46abbb Jul 17 01:24:37.139331 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:37.139315 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 01:24:37.740303 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:37.740262 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-685db4888-2swfm" event={"ID":"0b071c93-8839-46d9-a60e-7a79decb8e10","Type":"ContainerStarted","Data":"153d996501891126509429802f22ff4c3fcaeb53170b73f1a44a4e868e1e6cd7"} Jul 17 01:24:37.740303 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:37.740307 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-685db4888-2swfm" event={"ID":"0b071c93-8839-46d9-a60e-7a79decb8e10","Type":"ContainerStarted","Data":"895d32034fd933d89cc74745c82ca26f382370832c1ff467027ad75e6d46abbb"} Jul 17 01:24:37.756536 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:37.756477 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-685db4888-2swfm" podStartSLOduration=1.326331307 podStartE2EDuration="1.756459819s" totalImagesPullingTime="430.128512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:24:37.754585295 +0000 UTC m=+644.132509703" watchObservedRunningTime="2026-07-17 01:24:37.756459819 +0000 UTC m=+644.134384279" Jul 17 01:24:37.781550 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:37.781514 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bb5679577-4rmnf"] Jul 17 01:24:37.781945 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:37.781755 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5bb5679577-4rmnf" podUID="5493e9ef-22ea-4ced-8802-73bdd1dd6958" containerName="authorino" containerID="cri-o://f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88" gracePeriod=30 Jul 17 01:24:38.061770 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.061745 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:24:38.118657 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.118571 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5493e9ef-22ea-4ced-8802-73bdd1dd6958-tls-cert\") pod \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\" (UID: \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\") " Jul 17 01:24:38.118657 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.118609 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmpzs\" (UniqueName: \"kubernetes.io/projected/5493e9ef-22ea-4ced-8802-73bdd1dd6958-kube-api-access-cmpzs\") pod \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\" (UID: \"5493e9ef-22ea-4ced-8802-73bdd1dd6958\") " Jul 17 01:24:38.120874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.120839 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5493e9ef-22ea-4ced-8802-73bdd1dd6958-kube-api-access-cmpzs" (OuterVolumeSpecName: "kube-api-access-cmpzs") pod "5493e9ef-22ea-4ced-8802-73bdd1dd6958" (UID: "5493e9ef-22ea-4ced-8802-73bdd1dd6958"). InnerVolumeSpecName "kube-api-access-cmpzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:24:38.141526 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.141490 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5493e9ef-22ea-4ced-8802-73bdd1dd6958-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5493e9ef-22ea-4ced-8802-73bdd1dd6958" (UID: "5493e9ef-22ea-4ced-8802-73bdd1dd6958"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:24:38.219880 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.219840 2564 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5493e9ef-22ea-4ced-8802-73bdd1dd6958-tls-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:24:38.219880 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.219873 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmpzs\" (UniqueName: \"kubernetes.io/projected/5493e9ef-22ea-4ced-8802-73bdd1dd6958-kube-api-access-cmpzs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:24:38.747241 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.747200 2564 generic.go:356] "Generic (PLEG): container finished" podID="5493e9ef-22ea-4ced-8802-73bdd1dd6958" containerID="f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88" exitCode=0 Jul 17 01:24:38.747241 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.747243 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bb5679577-4rmnf" Jul 17 01:24:38.747800 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.747270 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bb5679577-4rmnf" event={"ID":"5493e9ef-22ea-4ced-8802-73bdd1dd6958","Type":"ContainerDied","Data":"f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88"} Jul 17 01:24:38.747800 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.747312 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bb5679577-4rmnf" event={"ID":"5493e9ef-22ea-4ced-8802-73bdd1dd6958","Type":"ContainerDied","Data":"3ab1575b326df8c1c29953716aa118552f287551076859f39d6dbe71b761f79a"} Jul 17 01:24:38.747800 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.747328 2564 scope.go:117] "RemoveContainer" containerID="f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88" Jul 17 01:24:38.758351 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.758330 2564 scope.go:117] "RemoveContainer" containerID="f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88" Jul 17 01:24:38.758649 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:24:38.758628 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88\": container with ID starting with f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88 not found: ID does not exist" containerID="f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88" Jul 17 01:24:38.758744 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.758657 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88"} err="failed to get container status \"f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88\": rpc error: code = NotFound desc = could not find container \"f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88\": container with ID starting with f6031ba61091096b02d94f554de2ff659966241740db37b36f544dcb93a0dc88 not found: ID does not exist" Jul 17 01:24:38.775441 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.775408 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bb5679577-4rmnf"] Jul 17 01:24:38.778913 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:38.778888 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5bb5679577-4rmnf"] Jul 17 01:24:40.135252 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:24:40.135224 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5493e9ef-22ea-4ced-8802-73bdd1dd6958" path="/var/lib/kubelet/pods/5493e9ef-22ea-4ced-8802-73bdd1dd6958/volumes" Jul 17 01:25:24.135291 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:24.135260 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-zbtwh"] Jul 17 01:25:24.135864 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:24.135525 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" podUID="a0f2cc71-54a6-48eb-af19-78d380f612b6" containerName="manager" containerID="cri-o://9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5" gracePeriod=30 Jul 17 01:25:24.404992 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:24.404969 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:25:24.562715 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:24.562686 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f2cc71-54a6-48eb-af19-78d380f612b6-webhook-cert\") pod \"a0f2cc71-54a6-48eb-af19-78d380f612b6\" (UID: \"a0f2cc71-54a6-48eb-af19-78d380f612b6\") " Jul 17 01:25:24.562923 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:24.562833 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f24r2\" (UniqueName: \"kubernetes.io/projected/a0f2cc71-54a6-48eb-af19-78d380f612b6-kube-api-access-f24r2\") pod \"a0f2cc71-54a6-48eb-af19-78d380f612b6\" (UID: \"a0f2cc71-54a6-48eb-af19-78d380f612b6\") " Jul 17 01:25:24.564813 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:24.564775 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0f2cc71-54a6-48eb-af19-78d380f612b6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a0f2cc71-54a6-48eb-af19-78d380f612b6" (UID: "a0f2cc71-54a6-48eb-af19-78d380f612b6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:25:24.564911 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:24.564844 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0f2cc71-54a6-48eb-af19-78d380f612b6-kube-api-access-f24r2" (OuterVolumeSpecName: "kube-api-access-f24r2") pod "a0f2cc71-54a6-48eb-af19-78d380f612b6" (UID: "a0f2cc71-54a6-48eb-af19-78d380f612b6"). InnerVolumeSpecName "kube-api-access-f24r2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:25:24.664500 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:24.664467 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f24r2\" (UniqueName: \"kubernetes.io/projected/a0f2cc71-54a6-48eb-af19-78d380f612b6-kube-api-access-f24r2\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:25:24.664500 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:24.664502 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0f2cc71-54a6-48eb-af19-78d380f612b6-webhook-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:25:25.021621 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:25.021523 2564 generic.go:356] "Generic (PLEG): container finished" podID="a0f2cc71-54a6-48eb-af19-78d380f612b6" containerID="9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5" exitCode=0 Jul 17 01:25:25.021621 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:25.021572 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" Jul 17 01:25:25.021832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:25.021624 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" event={"ID":"a0f2cc71-54a6-48eb-af19-78d380f612b6","Type":"ContainerDied","Data":"9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5"} Jul 17 01:25:25.021832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:25.021688 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-zbtwh" event={"ID":"a0f2cc71-54a6-48eb-af19-78d380f612b6","Type":"ContainerDied","Data":"b147dad9fc722f13800475bce5d481da766d90b4d9399b52d530da45b57b5cdc"} Jul 17 01:25:25.021832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:25.021711 2564 scope.go:117] "RemoveContainer" containerID="9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5" Jul 17 01:25:25.034433 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:25.033282 2564 scope.go:117] "RemoveContainer" containerID="9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5" Jul 17 01:25:25.034433 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:25:25.034387 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5\": container with ID starting with 9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5 not found: ID does not exist" containerID="9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5" Jul 17 01:25:25.034433 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:25.034418 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5"} err="failed to get container status \"9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5\": rpc error: code = NotFound desc = could not find container \"9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5\": container with ID starting with 9c4018a22edad439b2509ee782504e98a7f1c8583ea9da9c715eea4a5ba88ba5 not found: ID does not exist" Jul 17 01:25:25.043882 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:25.043863 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-zbtwh"] Jul 17 01:25:25.047341 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:25.047321 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-zbtwh"] Jul 17 01:25:26.133797 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:25:26.133763 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0f2cc71-54a6-48eb-af19-78d380f612b6" path="/var/lib/kubelet/pods/a0f2cc71-54a6-48eb-af19-78d380f612b6/volumes" Jul 17 01:26:25.846411 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.846374 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f988b7f6f-845dm"] Jul 17 01:26:25.846905 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.846800 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5493e9ef-22ea-4ced-8802-73bdd1dd6958" containerName="authorino" Jul 17 01:26:25.846905 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.846811 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5493e9ef-22ea-4ced-8802-73bdd1dd6958" containerName="authorino" Jul 17 01:26:25.846905 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.846820 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0f2cc71-54a6-48eb-af19-78d380f612b6" containerName="manager" Jul 17 01:26:25.846905 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.846825 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0f2cc71-54a6-48eb-af19-78d380f612b6" containerName="manager" Jul 17 01:26:25.846905 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.846896 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0f2cc71-54a6-48eb-af19-78d380f612b6" containerName="manager" Jul 17 01:26:25.846905 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.846905 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5493e9ef-22ea-4ced-8802-73bdd1dd6958" containerName="authorino" Jul 17 01:26:25.849931 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.849913 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:25.853508 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.853477 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 17 01:26:25.853654 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.853479 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w742h\"" Jul 17 01:26:25.853654 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.853481 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 17 01:26:25.856787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.856761 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-845dm"] Jul 17 01:26:25.951583 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.951549 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e767329a-f51b-4eae-afce-a49f4bb428f8-webhook-cert\") pod \"maas-controller-f988b7f6f-845dm\" (UID: \"e767329a-f51b-4eae-afce-a49f4bb428f8\") " pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:25.951752 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:25.951628 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5g9l\" (UniqueName: \"kubernetes.io/projected/e767329a-f51b-4eae-afce-a49f4bb428f8-kube-api-access-h5g9l\") pod \"maas-controller-f988b7f6f-845dm\" (UID: \"e767329a-f51b-4eae-afce-a49f4bb428f8\") " pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:26.053206 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:26.053147 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5g9l\" (UniqueName: \"kubernetes.io/projected/e767329a-f51b-4eae-afce-a49f4bb428f8-kube-api-access-h5g9l\") pod \"maas-controller-f988b7f6f-845dm\" (UID: \"e767329a-f51b-4eae-afce-a49f4bb428f8\") " pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:26.053417 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:26.053306 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e767329a-f51b-4eae-afce-a49f4bb428f8-webhook-cert\") pod \"maas-controller-f988b7f6f-845dm\" (UID: \"e767329a-f51b-4eae-afce-a49f4bb428f8\") " pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:26.055911 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:26.055886 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e767329a-f51b-4eae-afce-a49f4bb428f8-webhook-cert\") pod \"maas-controller-f988b7f6f-845dm\" (UID: \"e767329a-f51b-4eae-afce-a49f4bb428f8\") " pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:26.060653 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:26.060634 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5g9l\" (UniqueName: \"kubernetes.io/projected/e767329a-f51b-4eae-afce-a49f4bb428f8-kube-api-access-h5g9l\") pod \"maas-controller-f988b7f6f-845dm\" (UID: \"e767329a-f51b-4eae-afce-a49f4bb428f8\") " pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:26.162075 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:26.162038 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:26.304771 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:26.304738 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-845dm"] Jul 17 01:26:26.306359 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:26:26.306333 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode767329a_f51b_4eae_afce_a49f4bb428f8.slice/crio-22ed13d2ffcb445d76dda9de50ea663f13b0c5a7d1b203b210c2923276a0bd5b WatchSource:0}: Error finding container 22ed13d2ffcb445d76dda9de50ea663f13b0c5a7d1b203b210c2923276a0bd5b: Status 404 returned error can't find the container with id 22ed13d2ffcb445d76dda9de50ea663f13b0c5a7d1b203b210c2923276a0bd5b Jul 17 01:26:26.380185 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:26.380127 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-845dm" event={"ID":"e767329a-f51b-4eae-afce-a49f4bb428f8","Type":"ContainerStarted","Data":"22ed13d2ffcb445d76dda9de50ea663f13b0c5a7d1b203b210c2923276a0bd5b"} Jul 17 01:26:27.387182 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:27.387128 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-845dm" event={"ID":"e767329a-f51b-4eae-afce-a49f4bb428f8","Type":"ContainerStarted","Data":"705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800"} Jul 17 01:26:27.387630 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:27.387240 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:27.402719 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:27.402658 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f988b7f6f-845dm" podStartSLOduration=1.822524584 podStartE2EDuration="2.402641823s" totalImagesPullingTime="580.117239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:26:27.400615486 +0000 UTC m=+753.778539909" watchObservedRunningTime="2026-07-17 01:26:27.402641823 +0000 UTC m=+753.780566231" Jul 17 01:26:38.395625 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:38.395590 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:43.027337 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.027295 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-845dm"] Jul 17 01:26:43.027750 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.027519 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-f988b7f6f-845dm" podUID="e767329a-f51b-4eae-afce-a49f4bb428f8" containerName="manager" containerID="cri-o://705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800" gracePeriod=30 Jul 17 01:26:43.294123 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.294056 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:43.415934 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.415898 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e767329a-f51b-4eae-afce-a49f4bb428f8-webhook-cert\") pod \"e767329a-f51b-4eae-afce-a49f4bb428f8\" (UID: \"e767329a-f51b-4eae-afce-a49f4bb428f8\") " Jul 17 01:26:43.415934 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.415941 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5g9l\" (UniqueName: \"kubernetes.io/projected/e767329a-f51b-4eae-afce-a49f4bb428f8-kube-api-access-h5g9l\") pod \"e767329a-f51b-4eae-afce-a49f4bb428f8\" (UID: \"e767329a-f51b-4eae-afce-a49f4bb428f8\") " Jul 17 01:26:43.418337 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.418305 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e767329a-f51b-4eae-afce-a49f4bb428f8-kube-api-access-h5g9l" (OuterVolumeSpecName: "kube-api-access-h5g9l") pod "e767329a-f51b-4eae-afce-a49f4bb428f8" (UID: "e767329a-f51b-4eae-afce-a49f4bb428f8"). InnerVolumeSpecName "kube-api-access-h5g9l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:26:43.418468 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.418375 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e767329a-f51b-4eae-afce-a49f4bb428f8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e767329a-f51b-4eae-afce-a49f4bb428f8" (UID: "e767329a-f51b-4eae-afce-a49f4bb428f8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 01:26:43.481141 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.481100 2564 generic.go:356] "Generic (PLEG): container finished" podID="e767329a-f51b-4eae-afce-a49f4bb428f8" containerID="705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800" exitCode=0 Jul 17 01:26:43.481352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.481156 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-845dm" Jul 17 01:26:43.481352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.481189 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-845dm" event={"ID":"e767329a-f51b-4eae-afce-a49f4bb428f8","Type":"ContainerDied","Data":"705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800"} Jul 17 01:26:43.481352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.481255 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-845dm" event={"ID":"e767329a-f51b-4eae-afce-a49f4bb428f8","Type":"ContainerDied","Data":"22ed13d2ffcb445d76dda9de50ea663f13b0c5a7d1b203b210c2923276a0bd5b"} Jul 17 01:26:43.481352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.481282 2564 scope.go:117] "RemoveContainer" containerID="705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800" Jul 17 01:26:43.491483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.491455 2564 scope.go:117] "RemoveContainer" containerID="705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800" Jul 17 01:26:43.491763 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:26:43.491744 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800\": container with ID starting with 705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800 not found: ID does not exist" containerID="705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800" Jul 17 01:26:43.491826 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.491772 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800"} err="failed to get container status \"705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800\": rpc error: code = NotFound desc = could not find container \"705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800\": container with ID starting with 705ae10fec06855499ff9dbd88a68de32513a67ebd7b758fbe1a9dd4cd3cd800 not found: ID does not exist" Jul 17 01:26:43.502444 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.502417 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-845dm"] Jul 17 01:26:43.504960 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.504939 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-845dm"] Jul 17 01:26:43.516670 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.516636 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e767329a-f51b-4eae-afce-a49f4bb428f8-webhook-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:26:43.516670 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:43.516662 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5g9l\" (UniqueName: \"kubernetes.io/projected/e767329a-f51b-4eae-afce-a49f4bb428f8-kube-api-access-h5g9l\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:26:44.135771 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.135742 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e767329a-f51b-4eae-afce-a49f4bb428f8" path="/var/lib/kubelet/pods/e767329a-f51b-4eae-afce-a49f4bb428f8/volumes" Jul 17 01:26:44.270887 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.270853 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f988b7f6f-l87lh"] Jul 17 01:26:44.271334 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.271311 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e767329a-f51b-4eae-afce-a49f4bb428f8" containerName="manager" Jul 17 01:26:44.271334 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.271329 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e767329a-f51b-4eae-afce-a49f4bb428f8" containerName="manager" Jul 17 01:26:44.271493 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.271458 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e767329a-f51b-4eae-afce-a49f4bb428f8" containerName="manager" Jul 17 01:26:44.276111 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.276091 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:26:44.279146 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.279126 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 17 01:26:44.279395 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.279377 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 17 01:26:44.279587 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.279571 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w742h\"" Jul 17 01:26:44.281504 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.281481 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-l87lh"] Jul 17 01:26:44.424940 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.424897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43b71236-8373-4594-b50a-f0dd44ded33c-webhook-cert\") pod \"maas-controller-f988b7f6f-l87lh\" (UID: \"43b71236-8373-4594-b50a-f0dd44ded33c\") " pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:26:44.425134 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.424955 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj8wz\" (UniqueName: \"kubernetes.io/projected/43b71236-8373-4594-b50a-f0dd44ded33c-kube-api-access-nj8wz\") pod \"maas-controller-f988b7f6f-l87lh\" (UID: \"43b71236-8373-4594-b50a-f0dd44ded33c\") " pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:26:44.526235 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.526189 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43b71236-8373-4594-b50a-f0dd44ded33c-webhook-cert\") pod \"maas-controller-f988b7f6f-l87lh\" (UID: \"43b71236-8373-4594-b50a-f0dd44ded33c\") " pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:26:44.526421 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.526265 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj8wz\" (UniqueName: \"kubernetes.io/projected/43b71236-8373-4594-b50a-f0dd44ded33c-kube-api-access-nj8wz\") pod \"maas-controller-f988b7f6f-l87lh\" (UID: \"43b71236-8373-4594-b50a-f0dd44ded33c\") " pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:26:44.528739 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.528708 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43b71236-8373-4594-b50a-f0dd44ded33c-webhook-cert\") pod \"maas-controller-f988b7f6f-l87lh\" (UID: \"43b71236-8373-4594-b50a-f0dd44ded33c\") " pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:26:44.534202 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.534151 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj8wz\" (UniqueName: \"kubernetes.io/projected/43b71236-8373-4594-b50a-f0dd44ded33c-kube-api-access-nj8wz\") pod \"maas-controller-f988b7f6f-l87lh\" (UID: \"43b71236-8373-4594-b50a-f0dd44ded33c\") " pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:26:44.589750 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.589715 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:26:44.729610 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:44.729586 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f988b7f6f-l87lh"] Jul 17 01:26:44.731468 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:26:44.731437 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b71236_8373_4594_b50a_f0dd44ded33c.slice/crio-399f61fae27cd6e6eca6011a7750711f81f6a16e9df77b5f47e1822606921985 WatchSource:0}: Error finding container 399f61fae27cd6e6eca6011a7750711f81f6a16e9df77b5f47e1822606921985: Status 404 returned error can't find the container with id 399f61fae27cd6e6eca6011a7750711f81f6a16e9df77b5f47e1822606921985 Jul 17 01:26:45.495768 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:45.495728 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-l87lh" event={"ID":"43b71236-8373-4594-b50a-f0dd44ded33c","Type":"ContainerStarted","Data":"68d19b794a7ac393480a3d369980e2f5adc31abe3d8bddf0c31ec001a87b233d"} Jul 17 01:26:45.496137 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:45.495801 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f988b7f6f-l87lh" event={"ID":"43b71236-8373-4594-b50a-f0dd44ded33c","Type":"ContainerStarted","Data":"399f61fae27cd6e6eca6011a7750711f81f6a16e9df77b5f47e1822606921985"} Jul 17 01:26:45.496137 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:45.495927 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:26:45.511178 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:45.511116 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f988b7f6f-l87lh" podStartSLOduration=1.171730538 podStartE2EDuration="1.511099889s" totalImagesPullingTime="339.369351ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:26:45.509413089 +0000 UTC m=+771.887337499" watchObservedRunningTime="2026-07-17 01:26:45.511099889 +0000 UTC m=+771.889024335" Jul 17 01:26:56.502952 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:26:56.502920 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f988b7f6f-l87lh" Jul 17 01:28:54.091354 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:28:54.091265 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:28:54.091949 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:28:54.091555 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:31:38.747518 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:38.747481 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r"] Jul 17 01:31:38.748079 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:38.747688 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" podUID="55382604-6ad8-426a-8658-17794a5790ef" containerName="manager" containerID="cri-o://fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5" gracePeriod=10 Jul 17 01:31:42.509005 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:42.508981 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:31:42.601552 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:42.601455 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clf86\" (UniqueName: \"kubernetes.io/projected/55382604-6ad8-426a-8658-17794a5790ef-kube-api-access-clf86\") pod \"55382604-6ad8-426a-8658-17794a5790ef\" (UID: \"55382604-6ad8-426a-8658-17794a5790ef\") " Jul 17 01:31:42.601552 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:42.601512 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55382604-6ad8-426a-8658-17794a5790ef-extensions-socket-volume\") pod \"55382604-6ad8-426a-8658-17794a5790ef\" (UID: \"55382604-6ad8-426a-8658-17794a5790ef\") " Jul 17 01:31:42.601942 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:42.601910 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55382604-6ad8-426a-8658-17794a5790ef-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "55382604-6ad8-426a-8658-17794a5790ef" (UID: "55382604-6ad8-426a-8658-17794a5790ef"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:31:42.603683 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:42.603648 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55382604-6ad8-426a-8658-17794a5790ef-kube-api-access-clf86" (OuterVolumeSpecName: "kube-api-access-clf86") pod "55382604-6ad8-426a-8658-17794a5790ef" (UID: "55382604-6ad8-426a-8658-17794a5790ef"). InnerVolumeSpecName "kube-api-access-clf86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:31:42.702461 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:42.702423 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clf86\" (UniqueName: \"kubernetes.io/projected/55382604-6ad8-426a-8658-17794a5790ef-kube-api-access-clf86\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:31:42.702461 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:42.702456 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/55382604-6ad8-426a-8658-17794a5790ef-extensions-socket-volume\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:31:43.256138 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:43.256097 2564 generic.go:356] "Generic (PLEG): container finished" podID="55382604-6ad8-426a-8658-17794a5790ef" containerID="fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5" exitCode=0 Jul 17 01:31:43.256333 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:43.256178 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" Jul 17 01:31:43.256333 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:43.256194 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" event={"ID":"55382604-6ad8-426a-8658-17794a5790ef","Type":"ContainerDied","Data":"fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5"} Jul 17 01:31:43.256333 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:43.256263 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r" event={"ID":"55382604-6ad8-426a-8658-17794a5790ef","Type":"ContainerDied","Data":"645a136ccafb4389a634935a87df746ad298c52e048a38490f9915622eb04233"} Jul 17 01:31:43.256333 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:43.256292 2564 scope.go:117] "RemoveContainer" containerID="fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5" Jul 17 01:31:43.266599 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:43.266580 2564 scope.go:117] "RemoveContainer" containerID="fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5" Jul 17 01:31:43.266851 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:31:43.266832 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5\": container with ID starting with fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5 not found: ID does not exist" containerID="fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5" Jul 17 01:31:43.266891 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:43.266860 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5"} err="failed to get container status \"fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5\": rpc error: code = NotFound desc = could not find container \"fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5\": container with ID starting with fab2e19bf57f89fd29285d50e4662c6499cc7f46ab1f0266387cf4466cd2e5e5 not found: ID does not exist" Jul 17 01:31:43.280183 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:43.280143 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r"] Jul 17 01:31:43.284016 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:43.283995 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-h5m9r"] Jul 17 01:31:44.134050 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:31:44.134016 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55382604-6ad8-426a-8658-17794a5790ef" path="/var/lib/kubelet/pods/55382604-6ad8-426a-8658-17794a5790ef/volumes" Jul 17 01:32:39.888586 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:39.888537 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd"] Jul 17 01:32:39.891138 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:39.889119 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55382604-6ad8-426a-8658-17794a5790ef" containerName="manager" Jul 17 01:32:39.891138 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:39.889135 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="55382604-6ad8-426a-8658-17794a5790ef" containerName="manager" Jul 17 01:32:39.891138 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:39.889230 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="55382604-6ad8-426a-8658-17794a5790ef" containerName="manager" Jul 17 01:32:39.892276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:39.892255 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:32:39.894816 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:39.894793 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2x5dk\"" Jul 17 01:32:39.908413 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:39.908377 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd"] Jul 17 01:32:39.937575 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:39.937527 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7119e54a-0d4d-4e06-a7bc-1751d08a5f62-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tm2fd\" (UID: \"7119e54a-0d4d-4e06-a7bc-1751d08a5f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:32:39.937776 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:39.937602 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qhnv\" (UniqueName: \"kubernetes.io/projected/7119e54a-0d4d-4e06-a7bc-1751d08a5f62-kube-api-access-6qhnv\") pod \"kuadrant-operator-controller-manager-86878c769f-tm2fd\" (UID: \"7119e54a-0d4d-4e06-a7bc-1751d08a5f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:32:40.038977 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.038932 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7119e54a-0d4d-4e06-a7bc-1751d08a5f62-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tm2fd\" (UID: \"7119e54a-0d4d-4e06-a7bc-1751d08a5f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:32:40.039223 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.039001 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qhnv\" (UniqueName: \"kubernetes.io/projected/7119e54a-0d4d-4e06-a7bc-1751d08a5f62-kube-api-access-6qhnv\") pod \"kuadrant-operator-controller-manager-86878c769f-tm2fd\" (UID: \"7119e54a-0d4d-4e06-a7bc-1751d08a5f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:32:40.039379 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.039356 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7119e54a-0d4d-4e06-a7bc-1751d08a5f62-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tm2fd\" (UID: \"7119e54a-0d4d-4e06-a7bc-1751d08a5f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:32:40.049153 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.049125 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qhnv\" (UniqueName: \"kubernetes.io/projected/7119e54a-0d4d-4e06-a7bc-1751d08a5f62-kube-api-access-6qhnv\") pod \"kuadrant-operator-controller-manager-86878c769f-tm2fd\" (UID: \"7119e54a-0d4d-4e06-a7bc-1751d08a5f62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:32:40.202759 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.202722 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:32:40.357559 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.357527 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd"] Jul 17 01:32:40.359234 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:32:40.359208 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7119e54a_0d4d_4e06_a7bc_1751d08a5f62.slice/crio-fd9f74070ca7ba218fdce9809836107217592a8728c2d74a16b0f520efaa7653 WatchSource:0}: Error finding container fd9f74070ca7ba218fdce9809836107217592a8728c2d74a16b0f520efaa7653: Status 404 returned error can't find the container with id fd9f74070ca7ba218fdce9809836107217592a8728c2d74a16b0f520efaa7653 Jul 17 01:32:40.580464 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.580354 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" event={"ID":"7119e54a-0d4d-4e06-a7bc-1751d08a5f62","Type":"ContainerStarted","Data":"7a622f142c6cd74bbd5a48c31e7aec06052b9b3973a467d6738dfc8b945427ab"} Jul 17 01:32:40.580464 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.580418 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" event={"ID":"7119e54a-0d4d-4e06-a7bc-1751d08a5f62","Type":"ContainerStarted","Data":"fd9f74070ca7ba218fdce9809836107217592a8728c2d74a16b0f520efaa7653"} Jul 17 01:32:40.580668 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.580536 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:32:40.609010 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:40.608956 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" podStartSLOduration=1.608939525 podStartE2EDuration="1.608939525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:32:40.607360276 +0000 UTC m=+1126.985284685" watchObservedRunningTime="2026-07-17 01:32:40.608939525 +0000 UTC m=+1126.986863935" Jul 17 01:32:51.589578 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:32:51.589545 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tm2fd" Jul 17 01:33:54.133254 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:33:54.133225 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:33:54.137855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:33:54.137827 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:38:54.171234 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:38:54.171210 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:38:54.175737 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:38:54.175715 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:39:12.979957 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:12.979896 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59"] Jul 17 01:39:12.984600 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:12.984580 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:12.987692 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:12.987662 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d7122b7f-openshift-default-dockercfg-fxbt4\"" Jul 17 01:39:13.001934 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.001902 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59"] Jul 17 01:39:13.114434 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.114392 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-credential-socket\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.114625 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.114475 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-podinfo\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.114625 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.114512 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-envoy\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.114625 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.114538 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-data\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.114625 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.114563 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-token\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.114625 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.114598 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-socket\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.114845 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.114686 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-certs\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.114845 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.114725 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8lcg\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-kube-api-access-v8lcg\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.114845 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.114748 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istiod-ca-cert\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.215535 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.215485 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8lcg\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-kube-api-access-v8lcg\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.215535 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.215542 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istiod-ca-cert\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.215814 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.215590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-credential-socket\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.215814 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.215655 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-podinfo\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.215814 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.215701 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-envoy\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.215814 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.215721 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-data\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.215814 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.215743 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-token\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.215814 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.215796 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-socket\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.216191 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.215881 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-certs\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.216191 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.216113 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-credential-socket\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.216191 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.216148 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-data\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.216348 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.216247 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-certs\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.216414 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.216391 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-socket\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.216686 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.216669 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istiod-ca-cert\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.218266 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.218236 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-envoy\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.218502 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.218481 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-podinfo\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.223862 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.223838 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8lcg\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-kube-api-access-v8lcg\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.224095 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.224073 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-token\") pod \"e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.295536 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.295434 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:13.445675 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.445645 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59"] Jul 17 01:39:13.446757 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:39:13.446728 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb8b7a62_6a4c_47c6_81ca_ed83547e12ec.slice/crio-c66599fe206376ff19acacea19317803049e50a16c407d06559c8ec00e55c7a2 WatchSource:0}: Error finding container c66599fe206376ff19acacea19317803049e50a16c407d06559c8ec00e55c7a2: Status 404 returned error can't find the container with id c66599fe206376ff19acacea19317803049e50a16c407d06559c8ec00e55c7a2 Jul 17 01:39:13.448932 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.448887 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:39:13.449006 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.448990 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:39:13.449047 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.449035 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:39:13.876685 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.876645 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" event={"ID":"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec","Type":"ContainerStarted","Data":"94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684"} Jul 17 01:39:13.876870 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.876704 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" event={"ID":"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec","Type":"ContainerStarted","Data":"c66599fe206376ff19acacea19317803049e50a16c407d06559c8ec00e55c7a2"} Jul 17 01:39:13.900175 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:13.900118 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" podStartSLOduration=1.900101544 podStartE2EDuration="1.900101544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:39:13.897510089 +0000 UTC m=+1520.275434499" watchObservedRunningTime="2026-07-17 01:39:13.900101544 +0000 UTC m=+1520.278025951" Jul 17 01:39:14.296240 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:14.296204 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:14.303859 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:14.303822 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:14.882690 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:14.882665 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:14.883867 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:14.883846 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:25.956679 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:25.956588 2564 generic.go:356] "Generic (PLEG): container finished" podID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerID="94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684" exitCode=0 Jul 17 01:39:25.956679 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:25.956663 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" event={"ID":"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec","Type":"ContainerDied","Data":"94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684"} Jul 17 01:39:25.957271 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:25.957213 2564 scope.go:117] "RemoveContainer" containerID="94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684" Jul 17 01:39:25.958383 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:25.958354 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:39:25.958512 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:25.958423 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:39:25.958512 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:25.958457 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:39:26.964745 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:26.964713 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" event={"ID":"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec","Type":"ContainerStarted","Data":"6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb"} Jul 17 01:39:27.296718 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:27.296625 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:27.297462 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:27.297439 2564 patch_prober.go:28] interesting pod/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 17 01:39:27.297563 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:27.297489 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 17 01:39:28.296455 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:28.296413 2564 patch_prober.go:28] interesting pod/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 17 01:39:28.296858 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:28.296472 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 17 01:39:29.296455 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:29.296416 2564 patch_prober.go:28] interesting pod/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 17 01:39:29.296879 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:29.296476 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 17 01:39:29.882940 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:29.882906 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:30.296589 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:30.296550 2564 patch_prober.go:28] interesting pod/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 17 01:39:30.296994 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:30.296623 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 17 01:39:31.300782 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:31.300750 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:39:31.301966 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:39:31.301945 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:42:19.362458 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:19.362366 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59"] Jul 17 01:42:19.362978 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:19.362580 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" containerID="cri-o://6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb" gracePeriod=30 Jul 17 01:42:24.619396 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.619366 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:42:24.716895 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.716865 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-envoy\") pod \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " Jul 17 01:42:24.717077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.716912 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-data\") pod \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " Jul 17 01:42:24.717077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.716934 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-certs\") pod \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " Jul 17 01:42:24.717077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.716958 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-socket\") pod \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " Jul 17 01:42:24.717077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.716975 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-credential-socket\") pod \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " Jul 17 01:42:24.717077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717003 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istiod-ca-cert\") pod \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " Jul 17 01:42:24.717077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717033 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-token\") pod \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " Jul 17 01:42:24.717077 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717064 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8lcg\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-kube-api-access-v8lcg\") pod \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " Jul 17 01:42:24.717490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717125 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-podinfo\") pod \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\" (UID: \"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec\") " Jul 17 01:42:24.717490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717222 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" (UID: "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:42:24.717490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717291 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" (UID: "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:42:24.717648 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717493 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-credential-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:42:24.717648 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717509 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:42:24.717906 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717725 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" (UID: "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:42:24.717906 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717743 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" (UID: "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:42:24.717906 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.717767 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" (UID: "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:42:24.719336 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.719311 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" (UID: "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:42:24.719499 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.719475 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" (UID: "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 01:42:24.719724 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.719702 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" (UID: "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:42:24.719831 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.719809 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-kube-api-access-v8lcg" (OuterVolumeSpecName: "kube-api-access-v8lcg") pod "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" (UID: "bb8b7a62-6a4c-47c6-81ca-ed83547e12ec"). InnerVolumeSpecName "kube-api-access-v8lcg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:42:24.818411 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.818370 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:42:24.818411 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.818406 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8lcg\" (UniqueName: \"kubernetes.io/projected/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-kube-api-access-v8lcg\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:42:24.818411 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.818420 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-podinfo\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:42:24.818660 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.818433 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-envoy\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:42:24.818660 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.818443 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istio-data\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:42:24.818660 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.818453 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-workload-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:42:24.818660 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:24.818463 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec-istiod-ca-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:42:25.009989 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.009905 2564 generic.go:356] "Generic (PLEG): container finished" podID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerID="6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb" exitCode=0 Jul 17 01:42:25.009989 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.009955 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" event={"ID":"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec","Type":"ContainerDied","Data":"6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb"} Jul 17 01:42:25.009989 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.009966 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" Jul 17 01:42:25.010297 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.010008 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59" event={"ID":"bb8b7a62-6a4c-47c6-81ca-ed83547e12ec","Type":"ContainerDied","Data":"c66599fe206376ff19acacea19317803049e50a16c407d06559c8ec00e55c7a2"} Jul 17 01:42:25.010297 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.010032 2564 scope.go:117] "RemoveContainer" containerID="6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb" Jul 17 01:42:25.020493 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.020473 2564 scope.go:117] "RemoveContainer" containerID="94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684" Jul 17 01:42:25.029276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.029257 2564 scope.go:117] "RemoveContainer" containerID="6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb" Jul 17 01:42:25.029698 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:42:25.029561 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb\": container with ID starting with 6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb not found: ID does not exist" containerID="6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb" Jul 17 01:42:25.029698 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.029605 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb"} err="failed to get container status \"6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb\": rpc error: code = NotFound desc = could not find container \"6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb\": container with ID starting with 6b7b9fa8faf123c5fda1a403c17f5de0494b7f26b355d8af68c7fba36dcd3cdb not found: ID does not exist" Jul 17 01:42:25.029698 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.029630 2564 scope.go:117] "RemoveContainer" containerID="94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684" Jul 17 01:42:25.030133 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:42:25.030110 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684\": container with ID starting with 94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684 not found: ID does not exist" containerID="94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684" Jul 17 01:42:25.030239 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.030140 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684"} err="failed to get container status \"94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684\": rpc error: code = NotFound desc = could not find container \"94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684\": container with ID starting with 94a392ca09cf2de264d5bf4a5b7d41fe9faca9ebee7a3ac473e468a269dc1684 not found: ID does not exist" Jul 17 01:42:25.032963 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.032943 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59"] Jul 17 01:42:25.037014 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:25.036990 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d7122b7f-openshift-default-cd444d77c-mfv59"] Jul 17 01:42:26.139173 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:26.139137 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" path="/var/lib/kubelet/pods/bb8b7a62-6a4c-47c6-81ca-ed83547e12ec/volumes" Jul 17 01:42:31.516501 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.516437 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf"] Jul 17 01:42:31.517087 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.517062 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" Jul 17 01:42:31.517220 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.517091 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" Jul 17 01:42:31.517292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.517243 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" Jul 17 01:42:31.517292 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.517258 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" Jul 17 01:42:31.522277 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.522251 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.524933 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.524912 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-f0d41c05-gw-openshift-default-dockercfg-q24l6\"" Jul 17 01:42:31.533412 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.533387 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf"] Jul 17 01:42:31.586073 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.586036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-podinfo\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.586285 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.586096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-token\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.586285 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.586123 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-socket\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.586285 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.586186 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istiod-ca-cert\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.586462 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.586300 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-envoy\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.586462 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.586350 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwk8g\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-kube-api-access-rwk8g\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.586462 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.586427 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-data\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.586603 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.586465 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-credential-socket\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.586603 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.586534 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-certs\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.687517 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687480 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-data\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.687517 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-credential-socket\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.687787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-certs\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.687787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687581 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-podinfo\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.687787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687607 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-token\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.687787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687625 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-socket\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.687787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687659 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istiod-ca-cert\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.687787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687707 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-envoy\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.687787 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687741 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwk8g\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-kube-api-access-rwk8g\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.688126 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.687974 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-data\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.688200 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.688154 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-certs\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.688291 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.688271 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-credential-socket\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.688386 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.688359 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-socket\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.688585 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.688566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istiod-ca-cert\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.690333 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.690313 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-envoy\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.690614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.690592 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-podinfo\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.696569 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.696542 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-token\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.696665 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.696597 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwk8g\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-kube-api-access-rwk8g\") pod \"e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.834475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.834385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:31.979866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.979841 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf"] Jul 17 01:42:31.982396 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:42:31.982370 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19013508_ceb1_4cc5_8a8b_dd1afbb3fd9a.slice/crio-55b9dea6c371c47aae207a2d0f52fb44eeccfdbcdee2b8dfbf49e203cae6a13a WatchSource:0}: Error finding container 55b9dea6c371c47aae207a2d0f52fb44eeccfdbcdee2b8dfbf49e203cae6a13a: Status 404 returned error can't find the container with id 55b9dea6c371c47aae207a2d0f52fb44eeccfdbcdee2b8dfbf49e203cae6a13a Jul 17 01:42:31.984967 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.984936 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:42:31.985052 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.984998 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:42:31.985052 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:31.985026 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:42:32.057457 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:32.057422 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" event={"ID":"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a","Type":"ContainerStarted","Data":"55b9dea6c371c47aae207a2d0f52fb44eeccfdbcdee2b8dfbf49e203cae6a13a"} Jul 17 01:42:33.064682 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:33.064632 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" event={"ID":"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a","Type":"ContainerStarted","Data":"457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93"} Jul 17 01:42:33.084342 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:33.084295 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" podStartSLOduration=2.084279328 podStartE2EDuration="2.084279328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:42:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:42:33.083563336 +0000 UTC m=+1719.461487745" watchObservedRunningTime="2026-07-17 01:42:33.084279328 +0000 UTC m=+1719.462203735" Jul 17 01:42:33.835433 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:33.835392 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:33.841058 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:33.841030 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:34.073119 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:34.073081 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:34.074101 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:34.074082 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:44.135670 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:44.135629 2564 generic.go:356] "Generic (PLEG): container finished" podID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerID="457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93" exitCode=0 Jul 17 01:42:44.136085 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:44.135698 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" event={"ID":"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a","Type":"ContainerDied","Data":"457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93"} Jul 17 01:42:44.136085 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:44.136072 2564 scope.go:117] "RemoveContainer" containerID="457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93" Jul 17 01:42:44.137078 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:44.137055 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:42:44.137134 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:44.137118 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:42:44.137198 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:44.137182 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:42:45.143487 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:45.143450 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" event={"ID":"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a","Type":"ContainerStarted","Data":"38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b"} Jul 17 01:42:45.834950 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:45.834902 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:45.836419 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:45.836389 2564 patch_prober.go:28] interesting pod/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 17 01:42:45.836556 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:45.836443 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 17 01:42:46.835391 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:46.835353 2564 patch_prober.go:28] interesting pod/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 17 01:42:46.835800 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:46.835412 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 17 01:42:47.835465 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:47.835432 2564 patch_prober.go:28] interesting pod/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 17 01:42:47.835942 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:47.835914 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 17 01:42:48.835418 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:48.835373 2564 patch_prober.go:28] interesting pod/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Jul 17 01:42:48.835599 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:48.835437 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Jul 17 01:42:49.073280 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:49.073229 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:49.839403 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:49.839373 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:42:49.840473 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:42:49.840454 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:43:37.513822 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:37.513787 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf"] Jul 17 01:43:37.514389 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:37.514014 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" containerID="cri-o://38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b" gracePeriod=30 Jul 17 01:43:42.764829 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.764797 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:43:42.792653 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.792578 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-socket\") pod \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " Jul 17 01:43:42.792653 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.792614 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-envoy\") pod \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " Jul 17 01:43:42.792653 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.792638 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-token\") pod \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " Jul 17 01:43:42.792939 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.792675 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-credential-socket\") pod \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " Jul 17 01:43:42.792939 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.792710 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-certs\") pod \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " Jul 17 01:43:42.792939 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.792801 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istiod-ca-cert\") pod \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " Jul 17 01:43:42.792939 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.792862 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwk8g\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-kube-api-access-rwk8g\") pod \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " Jul 17 01:43:42.792939 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.792903 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-podinfo\") pod \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " Jul 17 01:43:42.792939 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.792929 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-data\") pod \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\" (UID: \"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a\") " Jul 17 01:43:42.793523 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.793495 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" (UID: "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:43:42.794439 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.793850 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" (UID: "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:43:42.794439 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.794339 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" (UID: "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:43:42.794439 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.794411 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" (UID: "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:43:42.795043 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.795015 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-data" (OuterVolumeSpecName: "istio-data") pod "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" (UID: "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:43:42.795588 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.795550 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" (UID: "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:43:42.795588 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.795564 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" (UID: "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:43:42.796738 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.796697 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" (UID: "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 01:43:42.797384 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.797362 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-kube-api-access-rwk8g" (OuterVolumeSpecName: "kube-api-access-rwk8g") pod "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" (UID: "19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a"). InnerVolumeSpecName "kube-api-access-rwk8g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:43:42.894274 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.894238 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwk8g\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-kube-api-access-rwk8g\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:43:42.894274 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.894270 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-podinfo\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:43:42.894274 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.894280 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-data\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:43:42.894498 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.894288 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:43:42.894498 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.894296 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-envoy\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:43:42.894498 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.894304 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:43:42.894498 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.894311 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-credential-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:43:42.894498 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.894319 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-workload-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:43:42.894498 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:42.894327 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a-istiod-ca-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:43:43.492921 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.492884 2564 generic.go:356] "Generic (PLEG): container finished" podID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerID="38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b" exitCode=0 Jul 17 01:43:43.493135 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.492958 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" Jul 17 01:43:43.493135 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.492974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" event={"ID":"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a","Type":"ContainerDied","Data":"38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b"} Jul 17 01:43:43.493135 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.493031 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf" event={"ID":"19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a","Type":"ContainerDied","Data":"55b9dea6c371c47aae207a2d0f52fb44eeccfdbcdee2b8dfbf49e203cae6a13a"} Jul 17 01:43:43.493135 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.493046 2564 scope.go:117] "RemoveContainer" containerID="38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b" Jul 17 01:43:43.503847 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.503821 2564 scope.go:117] "RemoveContainer" containerID="457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93" Jul 17 01:43:43.512529 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.512509 2564 scope.go:117] "RemoveContainer" containerID="38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b" Jul 17 01:43:43.512858 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:43:43.512831 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b\": container with ID starting with 38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b not found: ID does not exist" containerID="38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b" Jul 17 01:43:43.512972 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.512867 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b"} err="failed to get container status \"38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b\": rpc error: code = NotFound desc = could not find container \"38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b\": container with ID starting with 38a171861d132fffb9c948fc0f96b8bfbccd5e421423916ca1a039e3b904535b not found: ID does not exist" Jul 17 01:43:43.512972 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.512885 2564 scope.go:117] "RemoveContainer" containerID="457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93" Jul 17 01:43:43.513131 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:43:43.513105 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93\": container with ID starting with 457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93 not found: ID does not exist" containerID="457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93" Jul 17 01:43:43.513211 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.513135 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93"} err="failed to get container status \"457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93\": rpc error: code = NotFound desc = could not find container \"457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93\": container with ID starting with 457b73fd3f2b10aebf94ace75846c4d32c588bc9032d17b64a41f46ec75f5f93 not found: ID does not exist" Jul 17 01:43:43.515657 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.515634 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf"] Jul 17 01:43:43.520404 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:43.520381 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-f0d41c05-gw-openshift-default-5658bb6659-55jtf"] Jul 17 01:43:44.136206 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:44.136152 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" path="/var/lib/kubelet/pods/19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a/volumes" Jul 17 01:43:49.248150 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.248113 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx"] Jul 17 01:43:49.248815 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.248793 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" Jul 17 01:43:49.248884 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.248820 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" Jul 17 01:43:49.248884 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.248845 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" Jul 17 01:43:49.248884 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.248855 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb8b7a62-6a4c-47c6-81ca-ed83547e12ec" containerName="istio-proxy" Jul 17 01:43:49.249042 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.248897 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" Jul 17 01:43:49.249042 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.248907 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" Jul 17 01:43:49.249042 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.249017 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" Jul 17 01:43:49.249042 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.249031 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="19013508-ceb1-4cc5-8a8b-dd1afbb3fd9a" containerName="istio-proxy" Jul 17 01:43:49.254197 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.254148 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.256921 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.256890 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-2ef2c65d-openshift-default-dockercfg-7gw9c\"" Jul 17 01:43:49.262483 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.262416 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx"] Jul 17 01:43:49.355023 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.354982 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-token\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.355235 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.355079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istiod-ca-cert\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.355235 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.355206 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-certs\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.355341 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.355267 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-envoy\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.355341 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.355297 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-credential-socket\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.355341 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.355326 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-podinfo\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.355490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.355434 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-socket\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.355490 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.355470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-data\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.355603 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.355505 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7q8q\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-kube-api-access-x7q8q\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456316 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-certs\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456328 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-envoy\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-credential-socket\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456371 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-podinfo\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456430 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-socket\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456475 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-data\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456710 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456488 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7q8q\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-kube-api-access-x7q8q\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456710 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-token\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456710 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456559 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istiod-ca-cert\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456866 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456800 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-certs\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.456967 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456943 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-credential-socket\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.457029 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.456976 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-data\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.457092 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.457073 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-socket\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.457329 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.457313 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istiod-ca-cert\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.459513 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.459475 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-envoy\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.459672 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.459655 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-podinfo\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.465132 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.465110 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-token\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.466007 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.465981 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7q8q\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-kube-api-access-x7q8q\") pod \"e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.569631 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.569539 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:49.721916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.721885 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx"] Jul 17 01:43:49.723251 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:43:49.723224 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0cfb72c_0e0c_4037_a584_fe2aedfb79b2.slice/crio-cfd6b2821677226a4f0609b7c303b94822c607ffc385def9723f55aa09fe67e8 WatchSource:0}: Error finding container cfd6b2821677226a4f0609b7c303b94822c607ffc385def9723f55aa09fe67e8: Status 404 returned error can't find the container with id cfd6b2821677226a4f0609b7c303b94822c607ffc385def9723f55aa09fe67e8 Jul 17 01:43:49.725775 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.725741 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:43:49.725855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.725808 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:43:49.725855 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:49.725837 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:43:50.537273 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:50.537232 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" event={"ID":"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2","Type":"ContainerStarted","Data":"aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe"} Jul 17 01:43:50.537654 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:50.537288 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" event={"ID":"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2","Type":"ContainerStarted","Data":"cfd6b2821677226a4f0609b7c303b94822c607ffc385def9723f55aa09fe67e8"} Jul 17 01:43:50.559882 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:50.559824 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" podStartSLOduration=1.559806692 podStartE2EDuration="1.559806692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:43:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:43:50.55901466 +0000 UTC m=+1796.936939069" watchObservedRunningTime="2026-07-17 01:43:50.559806692 +0000 UTC m=+1796.937731100" Jul 17 01:43:50.570212 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:50.570179 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:50.575900 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:50.575875 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:51.542750 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:51.542716 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:51.543750 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:51.543732 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:43:54.208797 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:54.208769 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:43:54.215464 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:43:54.215440 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:44:05.625969 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:05.625933 2564 generic.go:356] "Generic (PLEG): container finished" podID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerID="aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe" exitCode=0 Jul 17 01:44:05.626532 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:05.626015 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" event={"ID":"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2","Type":"ContainerDied","Data":"aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe"} Jul 17 01:44:05.626624 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:05.626605 2564 scope.go:117] "RemoveContainer" containerID="aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe" Jul 17 01:44:05.627639 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:05.627615 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:44:05.627724 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:05.627711 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:44:05.627762 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:05.627756 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:44:06.542674 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:06.542625 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:44:06.635740 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:06.635693 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" event={"ID":"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2","Type":"ContainerStarted","Data":"06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b"} Jul 17 01:44:07.569854 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:07.569809 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:44:07.571111 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:07.571084 2564 patch_prober.go:28] interesting pod/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 17 01:44:07.571251 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:07.571144 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 17 01:44:08.570971 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:08.570926 2564 patch_prober.go:28] interesting pod/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 17 01:44:08.571491 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:08.570994 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 17 01:44:09.569914 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:09.569880 2564 patch_prober.go:28] interesting pod/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Jul 17 01:44:09.570090 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:09.569938 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Jul 17 01:44:10.574450 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:10.574412 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:44:10.574893 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:10.574809 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:44:10.575433 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:44:10.575414 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:48:54.249556 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:48:54.249523 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:48:54.256829 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:48:54.256808 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:49:55.746501 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:49:55.746414 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx"] Jul 17 01:49:55.746979 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:49:55.746634 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" containerID="cri-o://06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b" gracePeriod=30 Jul 17 01:50:01.009588 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.009559 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:50:01.113017 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.112974 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istiod-ca-cert\") pod \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " Jul 17 01:50:01.113231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113061 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-socket\") pod \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " Jul 17 01:50:01.113231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113104 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-credential-socket\") pod \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " Jul 17 01:50:01.113231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113128 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-data\") pod \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " Jul 17 01:50:01.113231 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113221 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-token\") pod \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " Jul 17 01:50:01.113471 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113263 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-envoy\") pod \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " Jul 17 01:50:01.113471 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113302 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-podinfo\") pod \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " Jul 17 01:50:01.113471 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113328 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7q8q\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-kube-api-access-x7q8q\") pod \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " Jul 17 01:50:01.113471 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113353 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-certs\") pod \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\" (UID: \"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2\") " Jul 17 01:50:01.113471 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113441 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" (UID: "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:50:01.113471 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113441 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" (UID: "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:50:01.113774 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113484 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" (UID: "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:50:01.113774 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113643 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" (UID: "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:50:01.113874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113787 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:50:01.113874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113807 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-credential-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:50:01.113874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113819 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-workload-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:50:01.113874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.113832 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istiod-ca-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:50:01.114082 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.114050 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" (UID: "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:50:01.115977 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.115947 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-kube-api-access-x7q8q" (OuterVolumeSpecName: "kube-api-access-x7q8q") pod "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" (UID: "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2"). InnerVolumeSpecName "kube-api-access-x7q8q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:50:01.116114 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.116094 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" (UID: "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:50:01.116253 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.116236 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" (UID: "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 01:50:01.116312 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.116269 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" (UID: "f0cfb72c-0e0c-4037-a584-fe2aedfb79b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:50:01.215111 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.215074 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-data\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:50:01.215111 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.215111 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:50:01.215385 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.215123 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-envoy\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:50:01.215385 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.215137 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-istio-podinfo\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:50:01.215385 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.215148 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7q8q\" (UniqueName: \"kubernetes.io/projected/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2-kube-api-access-x7q8q\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:50:01.683330 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.683293 2564 generic.go:356] "Generic (PLEG): container finished" podID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerID="06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b" exitCode=0 Jul 17 01:50:01.683540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.683355 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" Jul 17 01:50:01.683540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.683381 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" event={"ID":"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2","Type":"ContainerDied","Data":"06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b"} Jul 17 01:50:01.683540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.683440 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx" event={"ID":"f0cfb72c-0e0c-4037-a584-fe2aedfb79b2","Type":"ContainerDied","Data":"cfd6b2821677226a4f0609b7c303b94822c607ffc385def9723f55aa09fe67e8"} Jul 17 01:50:01.683540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.683463 2564 scope.go:117] "RemoveContainer" containerID="06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b" Jul 17 01:50:01.694076 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.694055 2564 scope.go:117] "RemoveContainer" containerID="aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe" Jul 17 01:50:01.703355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.703331 2564 scope.go:117] "RemoveContainer" containerID="06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b" Jul 17 01:50:01.703690 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:50:01.703666 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b\": container with ID starting with 06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b not found: ID does not exist" containerID="06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b" Jul 17 01:50:01.703755 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.703700 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b"} err="failed to get container status \"06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b\": rpc error: code = NotFound desc = could not find container \"06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b\": container with ID starting with 06c414a81b249d4a3b929339ff9d228a20380721aa1aa859e9012265cc410e3b not found: ID does not exist" Jul 17 01:50:01.703755 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.703720 2564 scope.go:117] "RemoveContainer" containerID="aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe" Jul 17 01:50:01.703958 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:50:01.703944 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe\": container with ID starting with aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe not found: ID does not exist" containerID="aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe" Jul 17 01:50:01.704005 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.703964 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe"} err="failed to get container status \"aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe\": rpc error: code = NotFound desc = could not find container \"aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe\": container with ID starting with aa7518816da36024cc74c3b2279c8aad8adb81fd3f097ee94c8afb3a5edf2ffe not found: ID does not exist" Jul 17 01:50:01.708823 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.708801 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx"] Jul 17 01:50:01.712157 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:01.712131 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-2ef2c65d-openshift-default-845494c7c7-h79lx"] Jul 17 01:50:02.134993 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:02.134903 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" path="/var/lib/kubelet/pods/f0cfb72c-0e0c-4037-a584-fe2aedfb79b2/volumes" Jul 17 01:50:07.637086 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.637054 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x"] Jul 17 01:50:07.637653 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.637635 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" Jul 17 01:50:07.637700 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.637656 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" Jul 17 01:50:07.637746 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.637741 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" Jul 17 01:50:07.642666 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.642647 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.645295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.645280 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-1e02ab63-openshift-default-dockercfg-h5dn9\"" Jul 17 01:50:07.651912 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.651875 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x"] Jul 17 01:50:07.675925 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.675897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-certs\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.676090 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.675957 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-socket\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.676090 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.675997 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-token\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.676090 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.676020 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-envoy\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.676090 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.676076 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-data\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.676317 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.676124 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istiod-ca-cert\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.676317 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.676207 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-podinfo\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.676317 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.676249 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-credential-socket\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.676442 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.676316 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z76t\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-kube-api-access-9z76t\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.777415 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777377 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z76t\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-kube-api-access-9z76t\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.777618 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777427 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-certs\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.777618 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777467 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-socket\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.777618 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777500 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-token\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.777618 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777523 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-envoy\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.777618 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-data\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.777618 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istiod-ca-cert\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.777957 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777664 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-podinfo\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.777957 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777707 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-credential-socket\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.778067 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.777988 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-certs\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.778067 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.778009 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-socket\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.778139 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.778115 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-data\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.778247 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.778225 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-credential-socket\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.778721 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.778694 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istiod-ca-cert\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.780040 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.780017 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-envoy\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.780282 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.780266 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-podinfo\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.785623 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.785603 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-token\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.785959 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.785932 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z76t\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-kube-api-access-9z76t\") pod \"e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:07.955820 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:07.955782 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:08.101639 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:08.101308 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x"] Jul 17 01:50:08.103586 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:50:08.103408 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda32e5a39_3a08_495f_a5fa_4d8f409dcc73.slice/crio-4a2f6eb909b73b210567f173306ce025c72d7de19e85aed7676e3c52406469c8 WatchSource:0}: Error finding container 4a2f6eb909b73b210567f173306ce025c72d7de19e85aed7676e3c52406469c8: Status 404 returned error can't find the container with id 4a2f6eb909b73b210567f173306ce025c72d7de19e85aed7676e3c52406469c8 Jul 17 01:50:08.106614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:08.106582 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:50:08.106727 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:08.106650 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:50:08.106727 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:08.106677 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:50:08.725853 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:08.725809 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" event={"ID":"a32e5a39-3a08-495f-a5fa-4d8f409dcc73","Type":"ContainerStarted","Data":"e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530"} Jul 17 01:50:08.726262 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:08.725898 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" event={"ID":"a32e5a39-3a08-495f-a5fa-4d8f409dcc73","Type":"ContainerStarted","Data":"4a2f6eb909b73b210567f173306ce025c72d7de19e85aed7676e3c52406469c8"} Jul 17 01:50:08.746898 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:08.746838 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" podStartSLOduration=1.746817944 podStartE2EDuration="1.746817944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:50:08.743452353 +0000 UTC m=+2175.121376760" watchObservedRunningTime="2026-07-17 01:50:08.746817944 +0000 UTC m=+2175.124742354" Jul 17 01:50:08.956763 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:08.956725 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:09.961895 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:09.961863 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:10.736829 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:10.736796 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:10.737823 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:10.737802 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:20.797840 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:20.797803 2564 generic.go:356] "Generic (PLEG): container finished" podID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerID="e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530" exitCode=0 Jul 17 01:50:20.798398 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:20.797880 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" event={"ID":"a32e5a39-3a08-495f-a5fa-4d8f409dcc73","Type":"ContainerDied","Data":"e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530"} Jul 17 01:50:20.798495 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:20.798477 2564 scope.go:117] "RemoveContainer" containerID="e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530" Jul 17 01:50:20.799537 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:20.799506 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:50:20.799627 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:20.799586 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:50:20.799741 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:20.799624 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:50:21.806027 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:21.805974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" event={"ID":"a32e5a39-3a08-495f-a5fa-4d8f409dcc73","Type":"ContainerStarted","Data":"aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd"} Jul 17 01:50:21.956261 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:21.956228 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:21.957220 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:21.957197 2564 patch_prober.go:28] interesting pod/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 17 01:50:21.957393 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:21.957373 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 17 01:50:22.956411 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:22.956383 2564 patch_prober.go:28] interesting pod/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 17 01:50:22.956847 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:22.956822 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 17 01:50:23.956366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:23.956334 2564 patch_prober.go:28] interesting pod/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 17 01:50:23.956538 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:23.956389 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 17 01:50:24.957137 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:24.957093 2564 patch_prober.go:28] interesting pod/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body= Jul 17 01:50:24.957559 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:24.957194 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" Jul 17 01:50:25.737067 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:25.737029 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:25.960434 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:25.960403 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:50:25.961409 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:50:25.961388 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:53:13.343536 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:13.343497 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x"] Jul 17 01:53:13.344055 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:13.343782 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" containerID="cri-o://aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd" gracePeriod=30 Jul 17 01:53:18.622018 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.621951 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:53:18.629836 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.629773 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-credential-socket\") pod \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " Jul 17 01:53:18.630029 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.629853 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istiod-ca-cert\") pod \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " Jul 17 01:53:18.630029 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.629918 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-socket\") pod \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " Jul 17 01:53:18.630029 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.629949 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-token\") pod \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " Jul 17 01:53:18.630029 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.629988 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-podinfo\") pod \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " Jul 17 01:53:18.630029 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.630020 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z76t\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-kube-api-access-9z76t\") pod \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " Jul 17 01:53:18.630342 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.630060 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-data\") pod \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " Jul 17 01:53:18.630342 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.630120 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a32e5a39-3a08-495f-a5fa-4d8f409dcc73" (UID: "a32e5a39-3a08-495f-a5fa-4d8f409dcc73"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:53:18.630342 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.630136 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-envoy\") pod \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " Jul 17 01:53:18.630570 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.630548 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-credential-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:53:18.630854 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.630818 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a32e5a39-3a08-495f-a5fa-4d8f409dcc73" (UID: "a32e5a39-3a08-495f-a5fa-4d8f409dcc73"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:53:18.631339 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.631303 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a32e5a39-3a08-495f-a5fa-4d8f409dcc73" (UID: "a32e5a39-3a08-495f-a5fa-4d8f409dcc73"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:53:18.631891 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.631836 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-data" (OuterVolumeSpecName: "istio-data") pod "a32e5a39-3a08-495f-a5fa-4d8f409dcc73" (UID: "a32e5a39-3a08-495f-a5fa-4d8f409dcc73"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:53:18.634599 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.634570 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a32e5a39-3a08-495f-a5fa-4d8f409dcc73" (UID: "a32e5a39-3a08-495f-a5fa-4d8f409dcc73"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:53:18.634599 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.634570 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-kube-api-access-9z76t" (OuterVolumeSpecName: "kube-api-access-9z76t") pod "a32e5a39-3a08-495f-a5fa-4d8f409dcc73" (UID: "a32e5a39-3a08-495f-a5fa-4d8f409dcc73"). InnerVolumeSpecName "kube-api-access-9z76t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:53:18.634992 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.634951 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a32e5a39-3a08-495f-a5fa-4d8f409dcc73" (UID: "a32e5a39-3a08-495f-a5fa-4d8f409dcc73"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 01:53:18.635089 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.634975 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-token" (OuterVolumeSpecName: "istio-token") pod "a32e5a39-3a08-495f-a5fa-4d8f409dcc73" (UID: "a32e5a39-3a08-495f-a5fa-4d8f409dcc73"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:53:18.731733 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.731697 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-certs\") pod \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\" (UID: \"a32e5a39-3a08-495f-a5fa-4d8f409dcc73\") " Jul 17 01:53:18.731975 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.731953 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a32e5a39-3a08-495f-a5fa-4d8f409dcc73" (UID: "a32e5a39-3a08-495f-a5fa-4d8f409dcc73"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:53:18.732024 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.731974 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-envoy\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:53:18.732024 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.731990 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istiod-ca-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:53:18.732024 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.732004 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:53:18.732024 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.732012 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:53:18.732146 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.732020 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-podinfo\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:53:18.732146 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.732040 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z76t\" (UniqueName: \"kubernetes.io/projected/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-kube-api-access-9z76t\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:53:18.732146 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.732049 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-istio-data\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:53:18.832554 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.832521 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a32e5a39-3a08-495f-a5fa-4d8f409dcc73-workload-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:53:18.835561 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.835532 2564 generic.go:356] "Generic (PLEG): container finished" podID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerID="aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd" exitCode=0 Jul 17 01:53:18.835696 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.835572 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" event={"ID":"a32e5a39-3a08-495f-a5fa-4d8f409dcc73","Type":"ContainerDied","Data":"aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd"} Jul 17 01:53:18.835696 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.835586 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" Jul 17 01:53:18.835696 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.835623 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x" event={"ID":"a32e5a39-3a08-495f-a5fa-4d8f409dcc73","Type":"ContainerDied","Data":"4a2f6eb909b73b210567f173306ce025c72d7de19e85aed7676e3c52406469c8"} Jul 17 01:53:18.835696 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.835660 2564 scope.go:117] "RemoveContainer" containerID="aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd" Jul 17 01:53:18.848360 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.848338 2564 scope.go:117] "RemoveContainer" containerID="e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530" Jul 17 01:53:18.857742 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.857720 2564 scope.go:117] "RemoveContainer" containerID="aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd" Jul 17 01:53:18.857996 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:53:18.857976 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd\": container with ID starting with aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd not found: ID does not exist" containerID="aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd" Jul 17 01:53:18.858065 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.858004 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd"} err="failed to get container status \"aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd\": rpc error: code = NotFound desc = could not find container \"aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd\": container with ID starting with aeb4f8f69b49f66c0a9b01967237628a64b7163461f135e7d4fe8f005fa518fd not found: ID does not exist" Jul 17 01:53:18.858065 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.858022 2564 scope.go:117] "RemoveContainer" containerID="e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530" Jul 17 01:53:18.858295 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:53:18.858280 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530\": container with ID starting with e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530 not found: ID does not exist" containerID="e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530" Jul 17 01:53:18.858344 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.858300 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530"} err="failed to get container status \"e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530\": rpc error: code = NotFound desc = could not find container \"e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530\": container with ID starting with e4627e8973145ce40134f80f69006780db18a4635e29ab974441124c0fbe3530 not found: ID does not exist" Jul 17 01:53:18.866934 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.866909 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x"] Jul 17 01:53:18.870377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:18.870354 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-1e02ab63-openshift-default-5ccffbf99f-k4m5x"] Jul 17 01:53:20.134008 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:20.133973 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" path="/var/lib/kubelet/pods/a32e5a39-3a08-495f-a5fa-4d8f409dcc73/volumes" Jul 17 01:53:54.285826 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:54.285797 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:53:54.295214 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:53:54.295189 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:54:59.757326 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.757293 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4"] Jul 17 01:54:59.757806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.757733 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" Jul 17 01:54:59.757806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.757745 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" Jul 17 01:54:59.757806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.757753 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" Jul 17 01:54:59.757806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.757758 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" Jul 17 01:54:59.757806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.757776 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" Jul 17 01:54:59.757806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.757781 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" Jul 17 01:54:59.757996 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.757854 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" Jul 17 01:54:59.757996 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.757864 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0cfb72c-0e0c-4037-a584-fe2aedfb79b2" containerName="istio-proxy" Jul 17 01:54:59.760971 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.760953 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:54:59.763374 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.763356 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ac7a9695-openshift-default-dockercfg-88vfs\"" Jul 17 01:54:59.774384 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.774356 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4"] Jul 17 01:54:59.899466 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.899425 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-envoy\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:54:59.899635 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.899506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-token\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:54:59.899635 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.899535 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-credential-socket\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:54:59.899635 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.899596 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e87377-247e-44f6-b4a7-a9a75b03dce1-istiod-ca-cert\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:54:59.899754 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.899659 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-socket\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:54:59.899754 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.899715 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-podinfo\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:54:59.899754 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.899744 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-data\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:54:59.899851 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.899773 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-certs\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:54:59.899851 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:54:59.899795 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6bv5\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-kube-api-access-k6bv5\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001232 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001157 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-envoy\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001437 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001253 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-token\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001437 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001279 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-credential-socket\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001437 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e87377-247e-44f6-b4a7-a9a75b03dce1-istiod-ca-cert\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001628 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001502 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-socket\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001628 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-podinfo\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001628 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001581 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-data\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001628 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001616 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-certs\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001653 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6bv5\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-kube-api-access-k6bv5\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001832 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001678 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-credential-socket\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001946 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001877 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-socket\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.001946 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.001922 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-data\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.002055 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.002029 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-certs\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.002141 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.002122 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e87377-247e-44f6-b4a7-a9a75b03dce1-istiod-ca-cert\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.003806 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.003780 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-envoy\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.004268 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.004250 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-podinfo\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.009230 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.009150 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-token\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.009421 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.009402 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6bv5\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-kube-api-access-k6bv5\") pod \"e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.072146 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.072111 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:00.223254 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.223230 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4"] Jul 17 01:55:00.225026 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:55:00.224998 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e87377_247e_44f6_b4a7_a9a75b03dce1.slice/crio-06c309be4a436f5c671e700fb1f95165e672fc4dcf99de36e1084edae4c22181 WatchSource:0}: Error finding container 06c309be4a436f5c671e700fb1f95165e672fc4dcf99de36e1084edae4c22181: Status 404 returned error can't find the container with id 06c309be4a436f5c671e700fb1f95165e672fc4dcf99de36e1084edae4c22181 Jul 17 01:55:00.227228 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.227198 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:55:00.227330 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.227267 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:55:00.227330 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.227298 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:55:00.413458 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.413419 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" event={"ID":"71e87377-247e-44f6-b4a7-a9a75b03dce1","Type":"ContainerStarted","Data":"d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867"} Jul 17 01:55:00.413641 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.413488 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" event={"ID":"71e87377-247e-44f6-b4a7-a9a75b03dce1","Type":"ContainerStarted","Data":"06c309be4a436f5c671e700fb1f95165e672fc4dcf99de36e1084edae4c22181"} Jul 17 01:55:00.436033 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:00.435970 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" podStartSLOduration=1.4359484820000001 podStartE2EDuration="1.435948482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:55:00.434549673 +0000 UTC m=+2466.812474082" watchObservedRunningTime="2026-07-17 01:55:00.435948482 +0000 UTC m=+2466.813872892" Jul 17 01:55:01.072429 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:01.072388 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:01.077263 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:01.077239 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:01.420796 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:01.420763 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:01.421738 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:01.421717 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:13.498818 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:13.498785 2564 generic.go:356] "Generic (PLEG): container finished" podID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerID="d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867" exitCode=0 Jul 17 01:55:13.499399 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:13.498863 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" event={"ID":"71e87377-247e-44f6-b4a7-a9a75b03dce1","Type":"ContainerDied","Data":"d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867"} Jul 17 01:55:13.499486 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:13.499407 2564 scope.go:117] "RemoveContainer" containerID="d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867" Jul 17 01:55:13.500512 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:13.500482 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:55:13.500589 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:13.500572 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:55:13.500633 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:13.500616 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:55:14.507661 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:14.506681 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" event={"ID":"71e87377-247e-44f6-b4a7-a9a75b03dce1","Type":"ContainerStarted","Data":"301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716"} Jul 17 01:55:15.073187 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:15.073136 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:15.074349 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:15.074327 2564 patch_prober.go:28] interesting pod/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 17 01:55:15.074407 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:15.074377 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 17 01:55:16.073393 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:16.073357 2564 patch_prober.go:28] interesting pod/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 17 01:55:16.073790 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:16.073414 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 17 01:55:16.420726 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:16.420693 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:17.073504 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:17.073467 2564 patch_prober.go:28] interesting pod/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 17 01:55:17.073984 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:17.073530 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 17 01:55:18.073217 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:18.073186 2564 patch_prober.go:28] interesting pod/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body= Jul 17 01:55:18.073392 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:18.073243 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" Jul 17 01:55:19.077493 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:19.077463 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:19.078403 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:19.078384 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:39.838108 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:39.838076 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4"] Jul 17 01:55:39.838680 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:39.838322 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" containerID="cri-o://301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716" gracePeriod=30 Jul 17 01:55:45.100100 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.100041 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:45.254144 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254112 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-envoy\") pod \"71e87377-247e-44f6-b4a7-a9a75b03dce1\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " Jul 17 01:55:45.254144 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254149 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e87377-247e-44f6-b4a7-a9a75b03dce1-istiod-ca-cert\") pod \"71e87377-247e-44f6-b4a7-a9a75b03dce1\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " Jul 17 01:55:45.254417 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254211 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-certs\") pod \"71e87377-247e-44f6-b4a7-a9a75b03dce1\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " Jul 17 01:55:45.254417 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254231 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-credential-socket\") pod \"71e87377-247e-44f6-b4a7-a9a75b03dce1\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " Jul 17 01:55:45.254417 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254259 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-socket\") pod \"71e87377-247e-44f6-b4a7-a9a75b03dce1\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " Jul 17 01:55:45.254417 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254298 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-podinfo\") pod \"71e87377-247e-44f6-b4a7-a9a75b03dce1\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " Jul 17 01:55:45.254417 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254346 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-token\") pod \"71e87377-247e-44f6-b4a7-a9a75b03dce1\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " Jul 17 01:55:45.254417 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254374 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6bv5\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-kube-api-access-k6bv5\") pod \"71e87377-247e-44f6-b4a7-a9a75b03dce1\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " Jul 17 01:55:45.254417 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254411 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-data\") pod \"71e87377-247e-44f6-b4a7-a9a75b03dce1\" (UID: \"71e87377-247e-44f6-b4a7-a9a75b03dce1\") " Jul 17 01:55:45.254755 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254540 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "71e87377-247e-44f6-b4a7-a9a75b03dce1" (UID: "71e87377-247e-44f6-b4a7-a9a75b03dce1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:55:45.254755 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254572 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "71e87377-247e-44f6-b4a7-a9a75b03dce1" (UID: "71e87377-247e-44f6-b4a7-a9a75b03dce1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:55:45.254755 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254683 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71e87377-247e-44f6-b4a7-a9a75b03dce1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "71e87377-247e-44f6-b4a7-a9a75b03dce1" (UID: "71e87377-247e-44f6-b4a7-a9a75b03dce1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:55:45.255082 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.254798 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "71e87377-247e-44f6-b4a7-a9a75b03dce1" (UID: "71e87377-247e-44f6-b4a7-a9a75b03dce1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:55:45.255383 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.255360 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/71e87377-247e-44f6-b4a7-a9a75b03dce1-istiod-ca-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:55:45.255496 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.255387 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:55:45.255496 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.255411 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-data" (OuterVolumeSpecName: "istio-data") pod "71e87377-247e-44f6-b4a7-a9a75b03dce1" (UID: "71e87377-247e-44f6-b4a7-a9a75b03dce1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:55:45.255496 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.255417 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-credential-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:55:45.255496 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.255439 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-workload-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:55:45.256685 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.256666 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "71e87377-247e-44f6-b4a7-a9a75b03dce1" (UID: "71e87377-247e-44f6-b4a7-a9a75b03dce1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:55:45.256887 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.256873 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-token" (OuterVolumeSpecName: "istio-token") pod "71e87377-247e-44f6-b4a7-a9a75b03dce1" (UID: "71e87377-247e-44f6-b4a7-a9a75b03dce1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:55:45.257089 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.257071 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-kube-api-access-k6bv5" (OuterVolumeSpecName: "kube-api-access-k6bv5") pod "71e87377-247e-44f6-b4a7-a9a75b03dce1" (UID: "71e87377-247e-44f6-b4a7-a9a75b03dce1"). InnerVolumeSpecName "kube-api-access-k6bv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:55:45.257473 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.257458 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "71e87377-247e-44f6-b4a7-a9a75b03dce1" (UID: "71e87377-247e-44f6-b4a7-a9a75b03dce1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 01:55:45.356192 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.356081 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-data\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:55:45.356192 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.356111 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-envoy\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:55:45.356192 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.356122 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-podinfo\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:55:45.356192 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.356131 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:55:45.356192 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.356139 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6bv5\" (UniqueName: \"kubernetes.io/projected/71e87377-247e-44f6-b4a7-a9a75b03dce1-kube-api-access-k6bv5\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:55:45.706860 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.706824 2564 generic.go:356] "Generic (PLEG): container finished" podID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerID="301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716" exitCode=0 Jul 17 01:55:45.707028 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.706868 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" Jul 17 01:55:45.707028 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.706906 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" event={"ID":"71e87377-247e-44f6-b4a7-a9a75b03dce1","Type":"ContainerDied","Data":"301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716"} Jul 17 01:55:45.707028 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.706960 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4" event={"ID":"71e87377-247e-44f6-b4a7-a9a75b03dce1","Type":"ContainerDied","Data":"06c309be4a436f5c671e700fb1f95165e672fc4dcf99de36e1084edae4c22181"} Jul 17 01:55:45.707028 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.706983 2564 scope.go:117] "RemoveContainer" containerID="301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716" Jul 17 01:55:45.716712 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.716689 2564 scope.go:117] "RemoveContainer" containerID="d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867" Jul 17 01:55:45.724983 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.724966 2564 scope.go:117] "RemoveContainer" containerID="301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716" Jul 17 01:55:45.725270 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:55:45.725252 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716\": container with ID starting with 301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716 not found: ID does not exist" containerID="301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716" Jul 17 01:55:45.725319 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.725280 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716"} err="failed to get container status \"301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716\": rpc error: code = NotFound desc = could not find container \"301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716\": container with ID starting with 301e81c6672e1d330b65c7faefa8763b35e63335d6233df1142fcc8340234716 not found: ID does not exist" Jul 17 01:55:45.725319 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.725296 2564 scope.go:117] "RemoveContainer" containerID="d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867" Jul 17 01:55:45.725538 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:55:45.725521 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867\": container with ID starting with d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867 not found: ID does not exist" containerID="d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867" Jul 17 01:55:45.725602 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.725543 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867"} err="failed to get container status \"d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867\": rpc error: code = NotFound desc = could not find container \"d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867\": container with ID starting with d6ddeb69ebace4e909f95c25756c341a0e390aa3cbfa5789bdb1082f06d9c867 not found: ID does not exist" Jul 17 01:55:45.737371 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.737337 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4"] Jul 17 01:55:45.737517 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:45.737390 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ac7a9695-openshift-default-cf57f8dbc-w9ls4"] Jul 17 01:55:46.134352 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:46.134276 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" path="/var/lib/kubelet/pods/71e87377-247e-44f6-b4a7-a9a75b03dce1/volumes" Jul 17 01:55:59.165794 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.165762 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr"] Jul 17 01:55:59.166440 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.166420 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" Jul 17 01:55:59.166535 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.166442 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" Jul 17 01:55:59.166535 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.166461 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" Jul 17 01:55:59.166535 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.166468 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" Jul 17 01:55:59.166682 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.166577 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" Jul 17 01:55:59.166682 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.166589 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a32e5a39-3a08-495f-a5fa-4d8f409dcc73" containerName="istio-proxy" Jul 17 01:55:59.166682 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.166598 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="71e87377-247e-44f6-b4a7-a9a75b03dce1" containerName="istio-proxy" Jul 17 01:55:59.169609 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.169589 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.172366 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.172347 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8c5b7a14-openshift-default-dockercfg-r2p2b\"" Jul 17 01:55:59.179986 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.179957 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr"] Jul 17 01:55:59.292296 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.292255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-credential-socket\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.292296 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.292293 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-token\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.292540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.292333 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istiod-ca-cert\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.292540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.292378 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxc2q\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-kube-api-access-mxc2q\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.292540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.292425 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-podinfo\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.292540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.292448 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-envoy\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.292540 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.292519 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-socket\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.292733 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.292577 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-data\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.292733 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.292599 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-certs\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.393867 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.393831 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istiod-ca-cert\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.393867 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.393869 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxc2q\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-kube-api-access-mxc2q\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394107 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.393992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-podinfo\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394107 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394028 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-envoy\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394107 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394058 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-socket\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394107 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-data\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394115 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-certs\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-credential-socket\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394355 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394253 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-token\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394563 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394518 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-data\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394563 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394548 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-socket\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394731 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394631 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-certs\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394731 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394721 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istiod-ca-cert\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.394810 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.394766 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-credential-socket\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.396500 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.396472 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-envoy\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.396657 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.396638 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-podinfo\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.401844 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.401824 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-token\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.402086 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.402069 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxc2q\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-kube-api-access-mxc2q\") pod \"e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.481656 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.481545 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:55:59.638098 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.638063 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr"] Jul 17 01:55:59.639955 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:55:59.639923 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a2078f5_213a_41b9_a5c6_adeb9ea9b0f1.slice/crio-d8e1a9b27a3ebe023d495e16d5b0b1e274095324762d0765096c5dd0b225d71c WatchSource:0}: Error finding container d8e1a9b27a3ebe023d495e16d5b0b1e274095324762d0765096c5dd0b225d71c: Status 404 returned error can't find the container with id d8e1a9b27a3ebe023d495e16d5b0b1e274095324762d0765096c5dd0b225d71c Jul 17 01:55:59.641703 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.641671 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:55:59.641804 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.641738 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:55:59.641804 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.641766 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:55:59.803556 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.803466 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" event={"ID":"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1","Type":"ContainerStarted","Data":"5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499"} Jul 17 01:55:59.803556 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.803523 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" event={"ID":"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1","Type":"ContainerStarted","Data":"d8e1a9b27a3ebe023d495e16d5b0b1e274095324762d0765096c5dd0b225d71c"} Jul 17 01:55:59.825393 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:55:59.825342 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" podStartSLOduration=0.825325667 podStartE2EDuration="825.325667ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:55:59.821202092 +0000 UTC m=+2526.199126499" watchObservedRunningTime="2026-07-17 01:55:59.825325667 +0000 UTC m=+2526.203250073" Jul 17 01:56:00.481939 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:00.481902 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:56:00.486810 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:00.486785 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:56:00.810087 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:00.810003 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:56:00.810993 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:00.810971 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:56:12.883579 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:12.883546 2564 generic.go:356] "Generic (PLEG): container finished" podID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerID="5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499" exitCode=0 Jul 17 01:56:12.883995 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:12.883596 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" event={"ID":"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1","Type":"ContainerDied","Data":"5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499"} Jul 17 01:56:12.884085 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:12.884065 2564 scope.go:117] "RemoveContainer" containerID="5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499" Jul 17 01:56:12.885191 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:12.885105 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:56:12.885305 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:12.885228 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:56:12.885305 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:12.885279 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:56:13.892405 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:13.892372 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" event={"ID":"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1","Type":"ContainerStarted","Data":"50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb"} Jul 17 01:56:14.482311 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:14.482271 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:56:14.483539 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:14.483513 2564 patch_prober.go:28] interesting pod/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 17 01:56:14.483647 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:14.483564 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 17 01:56:15.482877 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:15.482843 2564 patch_prober.go:28] interesting pod/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 17 01:56:15.483298 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:15.482894 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 17 01:56:15.810603 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:15.810499 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:56:16.482868 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:16.482825 2564 patch_prober.go:28] interesting pod/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 17 01:56:16.483137 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:16.482896 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 17 01:56:17.481951 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:17.481916 2564 patch_prober.go:28] interesting pod/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body= Jul 17 01:56:17.482132 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:17.481973 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" Jul 17 01:56:18.486335 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:18.486307 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:56:18.487499 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:56:18.487478 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:57:52.020498 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:52.020460 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr"] Jul 17 01:57:52.021008 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:52.020667 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" containerID="cri-o://50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb" gracePeriod=30 Jul 17 01:57:57.282260 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.282234 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:57:57.466685 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.466650 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-socket\") pod \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " Jul 17 01:57:57.466891 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.466705 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-token\") pod \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " Jul 17 01:57:57.466891 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.466736 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-certs\") pod \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " Jul 17 01:57:57.466891 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.466789 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-podinfo\") pod \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " Jul 17 01:57:57.466891 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.466835 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istiod-ca-cert\") pod \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " Jul 17 01:57:57.466891 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.466877 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-credential-socket\") pod \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " Jul 17 01:57:57.467156 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.466922 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-envoy\") pod \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " Jul 17 01:57:57.467156 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.466932 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" (UID: "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:57:57.467156 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.466960 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-data\") pod \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " Jul 17 01:57:57.467156 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.467008 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxc2q\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-kube-api-access-mxc2q\") pod \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\" (UID: \"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1\") " Jul 17 01:57:57.467156 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.467063 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" (UID: "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:57:57.467519 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.467256 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" (UID: "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 01:57:57.467519 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.467432 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istiod-ca-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:57:57.467519 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.467452 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:57:57.467519 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.467465 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-workload-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:57:57.467916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.467880 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" (UID: "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:57:57.468034 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.468005 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" (UID: "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:57:57.469276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.469252 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" (UID: "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 01:57:57.469276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.469265 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" (UID: "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 01:57:57.469669 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.469634 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" (UID: "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:57:57.469874 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.469848 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-kube-api-access-mxc2q" (OuterVolumeSpecName: "kube-api-access-mxc2q") pod "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" (UID: "2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1"). InnerVolumeSpecName "kube-api-access-mxc2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 01:57:57.506090 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.506051 2564 generic.go:356] "Generic (PLEG): container finished" podID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerID="50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb" exitCode=0 Jul 17 01:57:57.506276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.506130 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" Jul 17 01:57:57.506276 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.506145 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" event={"ID":"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1","Type":"ContainerDied","Data":"50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb"} Jul 17 01:57:57.506375 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.506279 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr" event={"ID":"2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1","Type":"ContainerDied","Data":"d8e1a9b27a3ebe023d495e16d5b0b1e274095324762d0765096c5dd0b225d71c"} Jul 17 01:57:57.506375 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.506313 2564 scope.go:117] "RemoveContainer" containerID="50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb" Jul 17 01:57:57.520568 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.520544 2564 scope.go:117] "RemoveContainer" containerID="5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499" Jul 17 01:57:57.530786 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.530769 2564 scope.go:117] "RemoveContainer" containerID="50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb" Jul 17 01:57:57.531079 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:57:57.531060 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb\": container with ID starting with 50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb not found: ID does not exist" containerID="50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb" Jul 17 01:57:57.531145 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.531086 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb"} err="failed to get container status \"50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb\": rpc error: code = NotFound desc = could not find container \"50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb\": container with ID starting with 50694b63be53d150cc30b24c67e29c1c16d21d2a6e0a3f7edd32f9ed5a5293cb not found: ID does not exist" Jul 17 01:57:57.531145 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.531105 2564 scope.go:117] "RemoveContainer" containerID="5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499" Jul 17 01:57:57.531360 ip-10-0-136-58 kubenswrapper[2564]: E0717 01:57:57.531341 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499\": container with ID starting with 5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499 not found: ID does not exist" containerID="5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499" Jul 17 01:57:57.531404 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.531369 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499"} err="failed to get container status \"5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499\": rpc error: code = NotFound desc = could not find container \"5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499\": container with ID starting with 5483d55a09b25fb3abb6436d7f5ae5e42ab6dabe3952779068641d16fae98499 not found: ID does not exist" Jul 17 01:57:57.544295 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.544263 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr"] Jul 17 01:57:57.552628 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.552595 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8c5b7a14-openshift-default-bd8c9d8cf-6vkjr"] Jul 17 01:57:57.568258 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.568226 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:57:57.568258 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.568257 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-podinfo\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:57:57.568439 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.568268 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-credential-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:57:57.568439 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.568279 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-envoy\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:57:57.568439 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.568289 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-istio-data\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:57:57.568439 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:57.568297 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxc2q\" (UniqueName: \"kubernetes.io/projected/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1-kube-api-access-mxc2q\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 01:57:58.135385 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:57:58.135348 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" path="/var/lib/kubelet/pods/2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1/volumes" Jul 17 01:58:46.703738 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.703638 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c"] Jul 17 01:58:46.704142 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.704109 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" Jul 17 01:58:46.704142 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.704122 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" Jul 17 01:58:46.704262 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.704147 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" Jul 17 01:58:46.704262 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.704152 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" Jul 17 01:58:46.704351 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.704269 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" Jul 17 01:58:46.707507 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.707487 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.710481 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.710465 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-089e15-openshift-default-dockercfg-v82gz\"" Jul 17 01:58:46.721414 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.721387 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c"] Jul 17 01:58:46.733779 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.733750 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-token\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.733928 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.733827 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-socket\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.733928 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.733845 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-data\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.734063 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.733958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-envoy\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.734123 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.734098 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmldm\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-kube-api-access-tmldm\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.734199 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.734149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-podinfo\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.734261 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.734210 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-certs\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.734261 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.734255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9eeb443f-56c2-44ef-97aa-356bc85aa954-istiod-ca-cert\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.734363 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.734282 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-credential-socket\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.834908 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.834872 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-socket\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.834908 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.834907 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-data\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835210 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.834932 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-envoy\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835210 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835050 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmldm\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-kube-api-access-tmldm\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835210 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835088 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-podinfo\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835210 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-certs\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835448 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835246 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9eeb443f-56c2-44ef-97aa-356bc85aa954-istiod-ca-cert\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835448 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835275 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-credential-socket\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835448 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835344 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-token\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835448 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835428 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-socket\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835664 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835582 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-certs\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835728 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835701 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-data\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.835791 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.835771 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-credential-socket\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.836330 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.836306 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9eeb443f-56c2-44ef-97aa-356bc85aa954-istiod-ca-cert\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.837562 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.837534 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-envoy\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.837835 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.837815 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-podinfo\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.844460 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.844438 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-token\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:46.844635 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:46.844618 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmldm\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-kube-api-access-tmldm\") pod \"e2e-inf-a-089e15-openshift-default-5957765745-bzc7c\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:47.018273 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:47.018151 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:47.173046 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:47.173013 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c"] Jul 17 01:58:47.174482 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:58:47.174454 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9eeb443f_56c2_44ef_97aa_356bc85aa954.slice/crio-1bc11f9946802058fe911f85bb9bdc9726a33c7f69a75c670003a6a645c405a4 WatchSource:0}: Error finding container 1bc11f9946802058fe911f85bb9bdc9726a33c7f69a75c670003a6a645c405a4: Status 404 returned error can't find the container with id 1bc11f9946802058fe911f85bb9bdc9726a33c7f69a75c670003a6a645c405a4 Jul 17 01:58:47.176692 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:47.176660 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:58:47.176785 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:47.176724 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:58:47.176785 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:47.176754 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:58:47.796768 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:47.796732 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" event={"ID":"9eeb443f-56c2-44ef-97aa-356bc85aa954","Type":"ContainerStarted","Data":"cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c"} Jul 17 01:58:47.797420 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:47.796786 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" event={"ID":"9eeb443f-56c2-44ef-97aa-356bc85aa954","Type":"ContainerStarted","Data":"1bc11f9946802058fe911f85bb9bdc9726a33c7f69a75c670003a6a645c405a4"} Jul 17 01:58:47.818925 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:47.818877 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" podStartSLOduration=1.818862615 podStartE2EDuration="1.818862615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:58:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:58:47.815632952 +0000 UTC m=+2694.193557358" watchObservedRunningTime="2026-07-17 01:58:47.818862615 +0000 UTC m=+2694.196787079" Jul 17 01:58:48.019067 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:48.019024 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:48.024570 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:48.024544 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:48.802982 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:48.802951 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:48.804037 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:48.804020 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:58:54.329900 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:54.329872 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:58:54.341208 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:58:54.341183 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 01:59:04.845377 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:04.845333 2564 patch_prober.go:28] interesting pod/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 17 01:59:04.845846 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:04.845396 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 17 01:59:05.904155 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:05.904120 2564 generic.go:356] "Generic (PLEG): container finished" podID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerID="cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c" exitCode=0 Jul 17 01:59:05.904586 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:05.904198 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" event={"ID":"9eeb443f-56c2-44ef-97aa-356bc85aa954","Type":"ContainerDied","Data":"cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c"} Jul 17 01:59:05.904668 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:05.904653 2564 scope.go:117] "RemoveContainer" containerID="cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c" Jul 17 01:59:05.905773 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:05.905739 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:59:05.905846 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:05.905830 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:59:05.905916 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:05.905878 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 01:59:06.917829 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:06.917782 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" event={"ID":"9eeb443f-56c2-44ef-97aa-356bc85aa954","Type":"ContainerStarted","Data":"3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2"} Jul 17 01:59:07.019284 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:07.019249 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:59:07.019531 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:07.019504 2564 patch_prober.go:28] interesting pod/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Jul 17 01:59:07.019705 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:07.019679 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Jul 17 01:59:08.019274 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:08.019228 2564 patch_prober.go:28] interesting pod/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Jul 17 01:59:08.019821 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:08.019299 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Jul 17 01:59:09.019242 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:09.019200 2564 patch_prober.go:28] interesting pod/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Jul 17 01:59:09.019469 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:09.019285 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Jul 17 01:59:10.018764 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:10.018715 2564 patch_prober.go:28] interesting pod/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body= Jul 17 01:59:10.018982 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:10.018786 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" Jul 17 01:59:11.023204 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.023150 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:59:11.023877 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.023857 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:59:11.024704 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.024681 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 01:59:11.071198 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.071146 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg"] Jul 17 01:59:11.071970 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.071948 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a2078f5-213a-41b9-a5c6-adeb9ea9b0f1" containerName="istio-proxy" Jul 17 01:59:11.074192 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.074142 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.077238 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.077216 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-089e15\"/\"test-model-089e15-kserve-self-signed-certs\"" Jul 17 01:59:11.077647 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.077616 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-089e15\"/\"openshift-service-ca.crt\"" Jul 17 01:59:11.077921 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.077902 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-089e15\"/\"default-dockercfg-xr22g\"" Jul 17 01:59:11.078214 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.077952 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-089e15\"/\"kube-root-ca.crt\"" Jul 17 01:59:11.089996 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.089969 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg"] Jul 17 01:59:11.180614 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.180578 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-home\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.180795 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.180630 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq94s\" (UniqueName: \"kubernetes.io/projected/c830bad5-c3b8-4a3c-9552-93542040e3e2-kube-api-access-dq94s\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.180795 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.180713 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-dshm\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.180795 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.180770 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c830bad5-c3b8-4a3c-9552-93542040e3e2-tls-certs\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.180904 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.180815 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-tmp-dir\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.180904 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.180831 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-model-cache\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.282154 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.282070 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-model-cache\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.282318 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.282197 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-home\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.282318 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.282222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq94s\" (UniqueName: \"kubernetes.io/projected/c830bad5-c3b8-4a3c-9552-93542040e3e2-kube-api-access-dq94s\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.282318 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.282251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-dshm\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.282492 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.282393 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c830bad5-c3b8-4a3c-9552-93542040e3e2-tls-certs\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.282492 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.282469 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-tmp-dir\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.282492 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.282484 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-model-cache\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.282641 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.282537 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-home\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.282781 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.282761 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-tmp-dir\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.284674 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.284652 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-dshm\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.285213 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.285191 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c830bad5-c3b8-4a3c-9552-93542040e3e2-tls-certs\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.289957 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.289930 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq94s\" (UniqueName: \"kubernetes.io/projected/c830bad5-c3b8-4a3c-9552-93542040e3e2-kube-api-access-dq94s\") pod \"test-model-089e15-kserve-5b5c486669-gtpvg\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.387846 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.387808 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:11.537514 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.537484 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg"] Jul 17 01:59:11.538818 ip-10-0-136-58 kubenswrapper[2564]: W0717 01:59:11.538792 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc830bad5_c3b8_4a3c_9552_93542040e3e2.slice/crio-ccb9f7a65df4166a0f4aeec374f2220d38e1ed473d91864204b8a4294f9fa1d7 WatchSource:0}: Error finding container ccb9f7a65df4166a0f4aeec374f2220d38e1ed473d91864204b8a4294f9fa1d7: Status 404 returned error can't find the container with id ccb9f7a65df4166a0f4aeec374f2220d38e1ed473d91864204b8a4294f9fa1d7 Jul 17 01:59:11.954465 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.954424 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" event={"ID":"c830bad5-c3b8-4a3c-9552-93542040e3e2","Type":"ContainerStarted","Data":"4365b677af1a102c2a3362d846fb97ebc0999760f42a79b7b3ce35223cb65cb6"} Jul 17 01:59:11.954651 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.954476 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" event={"ID":"c830bad5-c3b8-4a3c-9552-93542040e3e2","Type":"ContainerStarted","Data":"ccb9f7a65df4166a0f4aeec374f2220d38e1ed473d91864204b8a4294f9fa1d7"} Jul 17 01:59:11.976773 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:11.976720 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" podStartSLOduration=0.976702757 podStartE2EDuration="976.702757ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 01:59:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 01:59:11.975274516 +0000 UTC m=+2718.353198924" watchObservedRunningTime="2026-07-17 01:59:11.976702757 +0000 UTC m=+2718.354627165" Jul 17 01:59:21.388445 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:21.388406 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:21.388846 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:21.388466 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:21.401031 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:21.401000 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:22.030621 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:22.030594 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 01:59:46.079388 ip-10-0-136-58 kubenswrapper[2564]: I0717 01:59:46.079348 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg"] Jul 17 02:00:01.100708 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.100641 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" podUID="c830bad5-c3b8-4a3c-9552-93542040e3e2" containerName="main" containerID="cri-o://4365b677af1a102c2a3362d846fb97ebc0999760f42a79b7b3ce35223cb65cb6" gracePeriod=45 Jul 17 02:00:01.256946 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.256909 2564 generic.go:356] "Generic (PLEG): container finished" podID="c830bad5-c3b8-4a3c-9552-93542040e3e2" containerID="4365b677af1a102c2a3362d846fb97ebc0999760f42a79b7b3ce35223cb65cb6" exitCode=0 Jul 17 02:00:01.257126 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.256998 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" event={"ID":"c830bad5-c3b8-4a3c-9552-93542040e3e2","Type":"ContainerDied","Data":"4365b677af1a102c2a3362d846fb97ebc0999760f42a79b7b3ce35223cb65cb6"} Jul 17 02:00:01.360881 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.360804 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 02:00:01.457855 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.457827 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-home\") pod \"c830bad5-c3b8-4a3c-9552-93542040e3e2\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " Jul 17 02:00:01.458049 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.457867 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-dshm\") pod \"c830bad5-c3b8-4a3c-9552-93542040e3e2\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " Jul 17 02:00:01.458049 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.457903 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-model-cache\") pod \"c830bad5-c3b8-4a3c-9552-93542040e3e2\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " Jul 17 02:00:01.458049 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.457931 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-tmp-dir\") pod \"c830bad5-c3b8-4a3c-9552-93542040e3e2\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " Jul 17 02:00:01.458049 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.457990 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dq94s\" (UniqueName: \"kubernetes.io/projected/c830bad5-c3b8-4a3c-9552-93542040e3e2-kube-api-access-dq94s\") pod \"c830bad5-c3b8-4a3c-9552-93542040e3e2\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " Jul 17 02:00:01.458323 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.458063 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c830bad5-c3b8-4a3c-9552-93542040e3e2-tls-certs\") pod \"c830bad5-c3b8-4a3c-9552-93542040e3e2\" (UID: \"c830bad5-c3b8-4a3c-9552-93542040e3e2\") " Jul 17 02:00:01.458323 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.458155 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-home" (OuterVolumeSpecName: "home") pod "c830bad5-c3b8-4a3c-9552-93542040e3e2" (UID: "c830bad5-c3b8-4a3c-9552-93542040e3e2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:00:01.458323 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.458201 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-model-cache" (OuterVolumeSpecName: "model-cache") pod "c830bad5-c3b8-4a3c-9552-93542040e3e2" (UID: "c830bad5-c3b8-4a3c-9552-93542040e3e2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:00:01.458477 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.458320 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c830bad5-c3b8-4a3c-9552-93542040e3e2" (UID: "c830bad5-c3b8-4a3c-9552-93542040e3e2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:00:01.458477 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.458467 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-home\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:01.458572 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.458486 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-model-cache\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:01.458572 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.458528 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-tmp-dir\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:01.460377 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.460342 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c830bad5-c3b8-4a3c-9552-93542040e3e2-kube-api-access-dq94s" (OuterVolumeSpecName: "kube-api-access-dq94s") pod "c830bad5-c3b8-4a3c-9552-93542040e3e2" (UID: "c830bad5-c3b8-4a3c-9552-93542040e3e2"). InnerVolumeSpecName "kube-api-access-dq94s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:00:01.460491 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.460376 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c830bad5-c3b8-4a3c-9552-93542040e3e2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c830bad5-c3b8-4a3c-9552-93542040e3e2" (UID: "c830bad5-c3b8-4a3c-9552-93542040e3e2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 02:00:01.460491 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.460436 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-dshm" (OuterVolumeSpecName: "dshm") pod "c830bad5-c3b8-4a3c-9552-93542040e3e2" (UID: "c830bad5-c3b8-4a3c-9552-93542040e3e2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:00:01.559515 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.559481 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dq94s\" (UniqueName: \"kubernetes.io/projected/c830bad5-c3b8-4a3c-9552-93542040e3e2-kube-api-access-dq94s\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:01.559515 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.559509 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c830bad5-c3b8-4a3c-9552-93542040e3e2-tls-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:01.559515 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:01.559521 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c830bad5-c3b8-4a3c-9552-93542040e3e2-dshm\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:02.263957 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:02.263925 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" Jul 17 02:00:02.264438 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:02.263955 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg" event={"ID":"c830bad5-c3b8-4a3c-9552-93542040e3e2","Type":"ContainerDied","Data":"ccb9f7a65df4166a0f4aeec374f2220d38e1ed473d91864204b8a4294f9fa1d7"} Jul 17 02:00:02.264438 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:02.264010 2564 scope.go:117] "RemoveContainer" containerID="4365b677af1a102c2a3362d846fb97ebc0999760f42a79b7b3ce35223cb65cb6" Jul 17 02:00:02.282619 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:02.282596 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg"] Jul 17 02:00:02.284980 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:02.284962 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-089e15/test-model-089e15-kserve-5b5c486669-gtpvg"] Jul 17 02:00:04.136139 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:04.136096 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c830bad5-c3b8-4a3c-9552-93542040e3e2" path="/var/lib/kubelet/pods/c830bad5-c3b8-4a3c-9552-93542040e3e2/volumes" Jul 17 02:00:12.957809 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:12.957777 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c"] Jul 17 02:00:12.958241 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:12.957994 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" containerID="cri-o://3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2" gracePeriod=30 Jul 17 02:00:18.219450 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.219384 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 02:00:18.323422 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323389 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-podinfo\") pod \"9eeb443f-56c2-44ef-97aa-356bc85aa954\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " Jul 17 02:00:18.323422 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323423 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmldm\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-kube-api-access-tmldm\") pod \"9eeb443f-56c2-44ef-97aa-356bc85aa954\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " Jul 17 02:00:18.323688 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323459 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-certs\") pod \"9eeb443f-56c2-44ef-97aa-356bc85aa954\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " Jul 17 02:00:18.323688 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323493 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-data\") pod \"9eeb443f-56c2-44ef-97aa-356bc85aa954\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " Jul 17 02:00:18.323688 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323518 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-envoy\") pod \"9eeb443f-56c2-44ef-97aa-356bc85aa954\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " Jul 17 02:00:18.323688 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323554 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-token\") pod \"9eeb443f-56c2-44ef-97aa-356bc85aa954\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " Jul 17 02:00:18.323688 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323578 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-credential-socket\") pod \"9eeb443f-56c2-44ef-97aa-356bc85aa954\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " Jul 17 02:00:18.323960 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323856 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9eeb443f-56c2-44ef-97aa-356bc85aa954" (UID: "9eeb443f-56c2-44ef-97aa-356bc85aa954"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:00:18.323960 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323873 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9eeb443f-56c2-44ef-97aa-356bc85aa954" (UID: "9eeb443f-56c2-44ef-97aa-356bc85aa954"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:00:18.323960 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323910 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9eeb443f-56c2-44ef-97aa-356bc85aa954-istiod-ca-cert\") pod \"9eeb443f-56c2-44ef-97aa-356bc85aa954\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " Jul 17 02:00:18.324114 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.323990 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-socket\") pod \"9eeb443f-56c2-44ef-97aa-356bc85aa954\" (UID: \"9eeb443f-56c2-44ef-97aa-356bc85aa954\") " Jul 17 02:00:18.324404 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.324252 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9eeb443f-56c2-44ef-97aa-356bc85aa954-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9eeb443f-56c2-44ef-97aa-356bc85aa954" (UID: "9eeb443f-56c2-44ef-97aa-356bc85aa954"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 02:00:18.324503 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.324476 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9eeb443f-56c2-44ef-97aa-356bc85aa954" (UID: "9eeb443f-56c2-44ef-97aa-356bc85aa954"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:00:18.324635 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.324617 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:18.324635 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.324622 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-data" (OuterVolumeSpecName: "istio-data") pod "9eeb443f-56c2-44ef-97aa-356bc85aa954" (UID: "9eeb443f-56c2-44ef-97aa-356bc85aa954"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:00:18.324761 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.324642 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-workload-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:18.324761 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.324656 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-credential-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:18.324761 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.324664 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9eeb443f-56c2-44ef-97aa-356bc85aa954-istiod-ca-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:18.325857 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.325835 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9eeb443f-56c2-44ef-97aa-356bc85aa954" (UID: "9eeb443f-56c2-44ef-97aa-356bc85aa954"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 02:00:18.325857 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.325844 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-kube-api-access-tmldm" (OuterVolumeSpecName: "kube-api-access-tmldm") pod "9eeb443f-56c2-44ef-97aa-356bc85aa954" (UID: "9eeb443f-56c2-44ef-97aa-356bc85aa954"). InnerVolumeSpecName "kube-api-access-tmldm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:00:18.326261 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.326241 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9eeb443f-56c2-44ef-97aa-356bc85aa954" (UID: "9eeb443f-56c2-44ef-97aa-356bc85aa954"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:00:18.326331 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.326256 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-token" (OuterVolumeSpecName: "istio-token") pod "9eeb443f-56c2-44ef-97aa-356bc85aa954" (UID: "9eeb443f-56c2-44ef-97aa-356bc85aa954"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:00:18.365331 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.365297 2564 generic.go:356] "Generic (PLEG): container finished" podID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerID="3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2" exitCode=0 Jul 17 02:00:18.365485 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.365352 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" Jul 17 02:00:18.365485 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.365370 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" event={"ID":"9eeb443f-56c2-44ef-97aa-356bc85aa954","Type":"ContainerDied","Data":"3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2"} Jul 17 02:00:18.365485 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.365433 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c" event={"ID":"9eeb443f-56c2-44ef-97aa-356bc85aa954","Type":"ContainerDied","Data":"1bc11f9946802058fe911f85bb9bdc9726a33c7f69a75c670003a6a645c405a4"} Jul 17 02:00:18.365485 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.365455 2564 scope.go:117] "RemoveContainer" containerID="3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2" Jul 17 02:00:18.375725 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.375706 2564 scope.go:117] "RemoveContainer" containerID="cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c" Jul 17 02:00:18.383879 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.383864 2564 scope.go:117] "RemoveContainer" containerID="3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2" Jul 17 02:00:18.384123 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:00:18.384105 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2\": container with ID starting with 3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2 not found: ID does not exist" containerID="3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2" Jul 17 02:00:18.384190 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.384130 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2"} err="failed to get container status \"3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2\": rpc error: code = NotFound desc = could not find container \"3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2\": container with ID starting with 3e2f6a368d779003c0daa0b18cc4a2ab5cf2647d8da02474311d6e1d3dfa47b2 not found: ID does not exist" Jul 17 02:00:18.384190 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.384148 2564 scope.go:117] "RemoveContainer" containerID="cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c" Jul 17 02:00:18.384410 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:00:18.384389 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c\": container with ID starting with cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c not found: ID does not exist" containerID="cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c" Jul 17 02:00:18.384507 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.384413 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c"} err="failed to get container status \"cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c\": rpc error: code = NotFound desc = could not find container \"cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c\": container with ID starting with cdf12f86003871c94113e934757a5aa679fa6b0a2c6424f08ee85f0a7df5923c not found: ID does not exist" Jul 17 02:00:18.388511 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.388491 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c"] Jul 17 02:00:18.392406 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.392384 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-089e15-openshift-default-5957765745-bzc7c"] Jul 17 02:00:18.425937 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.425915 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-podinfo\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:18.425937 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.425939 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmldm\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-kube-api-access-tmldm\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:18.426078 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.425948 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-data\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:18.426078 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.425957 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-envoy\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:18.426078 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:18.425965 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9eeb443f-56c2-44ef-97aa-356bc85aa954-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:00:20.134837 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:20.134804 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" path="/var/lib/kubelet/pods/9eeb443f-56c2-44ef-97aa-356bc85aa954/volumes" Jul 17 02:00:39.950298 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.950255 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh"] Jul 17 02:00:39.950753 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.950736 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" Jul 17 02:00:39.950798 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.950756 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" Jul 17 02:00:39.950798 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.950764 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c830bad5-c3b8-4a3c-9552-93542040e3e2" containerName="main" Jul 17 02:00:39.950798 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.950770 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c830bad5-c3b8-4a3c-9552-93542040e3e2" containerName="main" Jul 17 02:00:39.950798 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.950787 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" Jul 17 02:00:39.950798 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.950792 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" Jul 17 02:00:39.950960 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.950868 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" Jul 17 02:00:39.950960 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.950876 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c830bad5-c3b8-4a3c-9552-93542040e3e2" containerName="main" Jul 17 02:00:39.953908 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.953883 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:39.956646 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.956619 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3413e7-openshift-default-dockercfg-fztvd\"" Jul 17 02:00:39.964501 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:39.964467 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh"] Jul 17 02:00:40.027765 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.027731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-certs\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.027765 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.027777 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-envoy\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.027992 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.027839 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f54bed-7f61-4f24-b668-2135aa825345-istiod-ca-cert\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.027992 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.027876 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvkrp\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-kube-api-access-kvkrp\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.027992 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.027916 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-istio-token\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.027992 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.027938 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-data\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.027992 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.027957 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-credential-socket\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.027992 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.027980 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f54bed-7f61-4f24-b668-2135aa825345-istio-podinfo\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.028254 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.028063 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-socket\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.128711 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.128657 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-envoy\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.128920 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.128726 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f54bed-7f61-4f24-b668-2135aa825345-istiod-ca-cert\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.128920 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.128753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvkrp\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-kube-api-access-kvkrp\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.128920 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.128781 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-istio-token\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.128920 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.128809 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-data\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.128920 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.128839 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-credential-socket\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.128920 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.128862 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f54bed-7f61-4f24-b668-2135aa825345-istio-podinfo\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.129275 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.128947 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-socket\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.129275 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.129005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-certs\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.129275 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.129255 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-credential-socket\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.129436 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.129293 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-data\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.129436 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.129323 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-certs\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.129524 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.129472 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-socket\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.129559 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.129516 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f54bed-7f61-4f24-b668-2135aa825345-istiod-ca-cert\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.131469 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.131444 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-envoy\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.131626 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.131609 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f54bed-7f61-4f24-b668-2135aa825345-istio-podinfo\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.138332 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.138307 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-istio-token\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.138432 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.138367 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvkrp\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-kube-api-access-kvkrp\") pod \"e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.266066 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.265974 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:40.410970 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.410939 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh"] Jul 17 02:00:40.412242 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:00:40.412196 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f54bed_7f61_4f24_b668_2135aa825345.slice/crio-8578ce1d27717b229705731287b5f7266eecc2abba2fc2f2981d514cc37c3fa9 WatchSource:0}: Error finding container 8578ce1d27717b229705731287b5f7266eecc2abba2fc2f2981d514cc37c3fa9: Status 404 returned error can't find the container with id 8578ce1d27717b229705731287b5f7266eecc2abba2fc2f2981d514cc37c3fa9 Jul 17 02:00:40.414473 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.414441 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:00:40.414559 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.414509 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:00:40.414559 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.414540 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:00:40.500795 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.500751 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" event={"ID":"68f54bed-7f61-4f24-b668-2135aa825345","Type":"ContainerStarted","Data":"4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67"} Jul 17 02:00:40.500952 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.500855 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" event={"ID":"68f54bed-7f61-4f24-b668-2135aa825345","Type":"ContainerStarted","Data":"8578ce1d27717b229705731287b5f7266eecc2abba2fc2f2981d514cc37c3fa9"} Jul 17 02:00:40.528197 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:40.528060 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" podStartSLOduration=1.52803822 podStartE2EDuration="1.52803822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:00:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:00:40.524629122 +0000 UTC m=+2806.902553554" watchObservedRunningTime="2026-07-17 02:00:40.52803822 +0000 UTC m=+2806.905962630" Jul 17 02:00:41.266696 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:41.266658 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:41.271738 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:41.271715 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:41.506582 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:41.506543 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:41.507450 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:41.507431 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:56.507933 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:56.507896 2564 patch_prober.go:28] interesting pod/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 17 02:00:56.508404 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:56.507960 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 17 02:00:56.598023 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:56.597990 2564 generic.go:356] "Generic (PLEG): container finished" podID="68f54bed-7f61-4f24-b668-2135aa825345" containerID="4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67" exitCode=0 Jul 17 02:00:56.598214 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:56.598067 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" event={"ID":"68f54bed-7f61-4f24-b668-2135aa825345","Type":"ContainerDied","Data":"4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67"} Jul 17 02:00:56.598508 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:56.598494 2564 scope.go:117] "RemoveContainer" containerID="4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67" Jul 17 02:00:56.599560 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:56.599534 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:00:56.599622 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:56.599604 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:00:56.599670 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:56.599636 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:00:57.606635 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:57.606577 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" event={"ID":"68f54bed-7f61-4f24-b668-2135aa825345","Type":"ContainerStarted","Data":"8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef"} Jul 17 02:00:58.266606 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:58.266567 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:00:58.266860 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:58.266833 2564 patch_prober.go:28] interesting pod/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 17 02:00:58.266941 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:58.266920 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 17 02:00:59.267480 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:59.267438 2564 patch_prober.go:28] interesting pod/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 17 02:00:59.267926 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:00:59.267512 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 17 02:01:00.267356 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:00.267317 2564 patch_prober.go:28] interesting pod/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 17 02:01:00.267546 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:00.267374 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 17 02:01:01.267021 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:01.266985 2564 patch_prober.go:28] interesting pod/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 17 02:01:01.267227 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:01.267041 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 17 02:01:02.271089 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:02.271057 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:01:02.271542 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:02.271452 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:01:02.272073 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:02.272055 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:01:28.961419 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:28.961382 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh"] Jul 17 02:01:28.961964 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:28.961599 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" containerID="cri-o://8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef" gracePeriod=30 Jul 17 02:01:32.274387 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:32.274345 2564 patch_prober.go:28] interesting pod/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 17 02:01:32.274775 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:32.274419 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 02:01:34.223510 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.223436 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:01:34.374208 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374152 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-certs\") pod \"68f54bed-7f61-4f24-b668-2135aa825345\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " Jul 17 02:01:34.374396 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374231 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvkrp\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-kube-api-access-kvkrp\") pod \"68f54bed-7f61-4f24-b668-2135aa825345\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " Jul 17 02:01:34.374396 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374261 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-istio-token\") pod \"68f54bed-7f61-4f24-b668-2135aa825345\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " Jul 17 02:01:34.374396 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374290 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f54bed-7f61-4f24-b668-2135aa825345-istiod-ca-cert\") pod \"68f54bed-7f61-4f24-b668-2135aa825345\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " Jul 17 02:01:34.374396 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374339 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-envoy\") pod \"68f54bed-7f61-4f24-b668-2135aa825345\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " Jul 17 02:01:34.374615 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374479 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-socket\") pod \"68f54bed-7f61-4f24-b668-2135aa825345\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " Jul 17 02:01:34.374615 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374543 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f54bed-7f61-4f24-b668-2135aa825345-istio-podinfo\") pod \"68f54bed-7f61-4f24-b668-2135aa825345\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " Jul 17 02:01:34.374615 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374573 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-credential-socket\") pod \"68f54bed-7f61-4f24-b668-2135aa825345\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " Jul 17 02:01:34.374763 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374700 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-data\") pod \"68f54bed-7f61-4f24-b668-2135aa825345\" (UID: \"68f54bed-7f61-4f24-b668-2135aa825345\") " Jul 17 02:01:34.374815 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374487 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "68f54bed-7f61-4f24-b668-2135aa825345" (UID: "68f54bed-7f61-4f24-b668-2135aa825345"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:01:34.374815 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374681 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "68f54bed-7f61-4f24-b668-2135aa825345" (UID: "68f54bed-7f61-4f24-b668-2135aa825345"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:01:34.374815 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374734 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/68f54bed-7f61-4f24-b668-2135aa825345-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "68f54bed-7f61-4f24-b668-2135aa825345" (UID: "68f54bed-7f61-4f24-b668-2135aa825345"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 02:01:34.374963 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.374893 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "68f54bed-7f61-4f24-b668-2135aa825345" (UID: "68f54bed-7f61-4f24-b668-2135aa825345"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:01:34.375282 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.375124 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:01:34.375523 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.375500 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-credential-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:01:34.375618 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.375531 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-workload-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:01:34.375618 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.375548 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f54bed-7f61-4f24-b668-2135aa825345-istiod-ca-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:01:34.375729 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.375689 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-data" (OuterVolumeSpecName: "istio-data") pod "68f54bed-7f61-4f24-b668-2135aa825345" (UID: "68f54bed-7f61-4f24-b668-2135aa825345"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:01:34.376758 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.376736 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-istio-token" (OuterVolumeSpecName: "istio-token") pod "68f54bed-7f61-4f24-b668-2135aa825345" (UID: "68f54bed-7f61-4f24-b668-2135aa825345"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:01:34.376841 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.376790 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/68f54bed-7f61-4f24-b668-2135aa825345-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "68f54bed-7f61-4f24-b668-2135aa825345" (UID: "68f54bed-7f61-4f24-b668-2135aa825345"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 02:01:34.377102 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.377083 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-kube-api-access-kvkrp" (OuterVolumeSpecName: "kube-api-access-kvkrp") pod "68f54bed-7f61-4f24-b668-2135aa825345" (UID: "68f54bed-7f61-4f24-b668-2135aa825345"). InnerVolumeSpecName "kube-api-access-kvkrp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:01:34.377210 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.377096 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "68f54bed-7f61-4f24-b668-2135aa825345" (UID: "68f54bed-7f61-4f24-b668-2135aa825345"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:01:34.476868 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.476792 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:01:34.476868 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.476818 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-envoy\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:01:34.476868 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.476826 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f54bed-7f61-4f24-b668-2135aa825345-istio-podinfo\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:01:34.476868 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.476834 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f54bed-7f61-4f24-b668-2135aa825345-istio-data\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:01:34.476868 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.476841 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvkrp\" (UniqueName: \"kubernetes.io/projected/68f54bed-7f61-4f24-b668-2135aa825345-kube-api-access-kvkrp\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:01:34.831845 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.831763 2564 generic.go:356] "Generic (PLEG): container finished" podID="68f54bed-7f61-4f24-b668-2135aa825345" containerID="8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef" exitCode=0 Jul 17 02:01:34.831845 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.831817 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" Jul 17 02:01:34.832033 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.831856 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" event={"ID":"68f54bed-7f61-4f24-b668-2135aa825345","Type":"ContainerDied","Data":"8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef"} Jul 17 02:01:34.832033 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.831914 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh" event={"ID":"68f54bed-7f61-4f24-b668-2135aa825345","Type":"ContainerDied","Data":"8578ce1d27717b229705731287b5f7266eecc2abba2fc2f2981d514cc37c3fa9"} Jul 17 02:01:34.832033 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.831947 2564 scope.go:117] "RemoveContainer" containerID="8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef" Jul 17 02:01:34.849928 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.849908 2564 scope.go:117] "RemoveContainer" containerID="4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67" Jul 17 02:01:34.860413 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.860387 2564 scope.go:117] "RemoveContainer" containerID="8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef" Jul 17 02:01:34.860800 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:01:34.860771 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef\": container with ID starting with 8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef not found: ID does not exist" containerID="8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef" Jul 17 02:01:34.860920 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.860811 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef"} err="failed to get container status \"8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef\": rpc error: code = NotFound desc = could not find container \"8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef\": container with ID starting with 8ee58f4e40810c23b42f7cd7f71d21ffda2d944fd556dd968ec9b3d32a42deef not found: ID does not exist" Jul 17 02:01:34.860920 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.860837 2564 scope.go:117] "RemoveContainer" containerID="4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67" Jul 17 02:01:34.861142 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:01:34.861122 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67\": container with ID starting with 4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67 not found: ID does not exist" containerID="4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67" Jul 17 02:01:34.861465 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.861150 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67"} err="failed to get container status \"4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67\": rpc error: code = NotFound desc = could not find container \"4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67\": container with ID starting with 4ad266dfe7e2382064ebfe2f9fcf3497b18d5432586d21ab84205b31aff72d67 not found: ID does not exist" Jul 17 02:01:34.861465 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.861268 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh"] Jul 17 02:01:34.863776 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:34.863756 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3413e7-openshift-default-556bbb465c-cgvkh"] Jul 17 02:01:36.134555 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:36.134521 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68f54bed-7f61-4f24-b668-2135aa825345" path="/var/lib/kubelet/pods/68f54bed-7f61-4f24-b668-2135aa825345/volumes" Jul 17 02:01:55.965675 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.965639 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw"] Jul 17 02:01:55.966190 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.966090 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" Jul 17 02:01:55.966190 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.966103 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" Jul 17 02:01:55.966190 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.966119 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" Jul 17 02:01:55.966190 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.966124 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" Jul 17 02:01:55.966452 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.966216 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" Jul 17 02:01:55.966452 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.966226 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9eeb443f-56c2-44ef-97aa-356bc85aa954" containerName="istio-proxy" Jul 17 02:01:55.966452 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.966233 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="68f54bed-7f61-4f24-b668-2135aa825345" containerName="istio-proxy" Jul 17 02:01:55.970987 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.970965 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:55.976213 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.976191 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-fd4ae7-openshift-default-dockercfg-trzck\"" Jul 17 02:01:55.985746 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:55.985715 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw"] Jul 17 02:01:56.062526 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.062492 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjb5f\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-kube-api-access-bjb5f\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.062526 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.062531 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-certs\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.062738 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.062552 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-socket\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.062738 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.062581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-podinfo\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.062738 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.062631 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-data\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.062738 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.062703 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-credential-socket\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.062738 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.062735 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff59dab7-2302-49dc-a390-661fc2568b3f-istiod-ca-cert\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.062903 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.062786 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-envoy\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.062903 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.062834 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-token\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163449 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjb5f\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-kube-api-access-bjb5f\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163659 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163464 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-certs\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163659 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163493 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-socket\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163659 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163521 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-podinfo\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163659 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-data\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163659 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163611 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-credential-socket\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163659 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163635 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff59dab7-2302-49dc-a390-661fc2568b3f-istiod-ca-cert\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163991 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163695 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-envoy\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163991 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163749 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-token\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.163991 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.163905 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-certs\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.164187 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.164114 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-credential-socket\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.164187 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.164138 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-socket\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.164187 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.164158 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-data\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.164492 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.164476 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff59dab7-2302-49dc-a390-661fc2568b3f-istiod-ca-cert\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.166144 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.166122 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-envoy\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.166258 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.166198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-podinfo\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.171935 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.171914 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-token\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.172023 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.171944 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjb5f\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-kube-api-access-bjb5f\") pod \"e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.281947 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.281855 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:56.427494 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.427464 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw"] Jul 17 02:01:56.428597 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:01:56.428572 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff59dab7_2302_49dc_a390_661fc2568b3f.slice/crio-b107a0432e413ed80b6112a47986ea667e6cf555e6545e9954fdc34007faa9fb WatchSource:0}: Error finding container b107a0432e413ed80b6112a47986ea667e6cf555e6545e9954fdc34007faa9fb: Status 404 returned error can't find the container with id b107a0432e413ed80b6112a47986ea667e6cf555e6545e9954fdc34007faa9fb Jul 17 02:01:56.430700 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.430668 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:01:56.430777 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.430761 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:01:56.430817 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.430797 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:01:56.962225 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.962185 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" event={"ID":"ff59dab7-2302-49dc-a390-661fc2568b3f","Type":"ContainerStarted","Data":"9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5"} Jul 17 02:01:56.962406 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.962247 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" event={"ID":"ff59dab7-2302-49dc-a390-661fc2568b3f","Type":"ContainerStarted","Data":"b107a0432e413ed80b6112a47986ea667e6cf555e6545e9954fdc34007faa9fb"} Jul 17 02:01:56.984974 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:56.984920 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" podStartSLOduration=1.984903741 podStartE2EDuration="1.984903741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:01:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:01:56.98341478 +0000 UTC m=+2883.361339191" watchObservedRunningTime="2026-07-17 02:01:56.984903741 +0000 UTC m=+2883.362828150" Jul 17 02:01:57.282646 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:57.282536 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:57.288243 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:57.288217 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:57.967482 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:57.967454 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:01:57.968458 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:01:57.968437 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:02:12.078308 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:12.078223 2564 generic.go:356] "Generic (PLEG): container finished" podID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerID="9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5" exitCode=0 Jul 17 02:02:12.078715 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:12.078298 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" event={"ID":"ff59dab7-2302-49dc-a390-661fc2568b3f","Type":"ContainerDied","Data":"9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5"} Jul 17 02:02:12.078758 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:12.078715 2564 scope.go:117] "RemoveContainer" containerID="9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5" Jul 17 02:02:12.079759 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:12.079733 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:02:12.079855 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:12.079796 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:02:12.079855 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:12.079829 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 02:02:12.967992 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:12.967952 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:02:13.085879 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:13.085847 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" event={"ID":"ff59dab7-2302-49dc-a390-661fc2568b3f","Type":"ContainerStarted","Data":"cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942"} Jul 17 02:02:13.282780 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:13.282676 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:02:13.283939 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:13.283914 2564 patch_prober.go:28] interesting pod/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 17 02:02:13.284064 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:13.283965 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 17 02:02:14.283379 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:14.283335 2564 patch_prober.go:28] interesting pod/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 17 02:02:14.283786 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:14.283397 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 17 02:02:15.282311 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:15.282271 2564 patch_prober.go:28] interesting pod/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 17 02:02:15.282528 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:15.282368 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 17 02:02:16.282485 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:16.282443 2564 patch_prober.go:28] interesting pod/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 17 02:02:16.282865 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:16.282507 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 17 02:02:17.286666 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:17.286636 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:02:17.287079 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:17.286937 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:02:17.287641 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:17.287624 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:02:23.990674 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:23.990640 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979"] Jul 17 02:02:23.993294 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:23.993273 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:23.996980 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:23.996958 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fd4ae7\"/\"kube-root-ca.crt\"" Jul 17 02:02:23.996980 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:23.996973 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fd4ae7\"/\"default-dockercfg-6bf8t\"" Jul 17 02:02:23.997191 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:23.996972 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fd4ae7\"/\"auth-test-model-fd4ae7-kserve-self-signed-certs\"" Jul 17 02:02:23.997191 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:23.997010 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fd4ae7\"/\"openshift-service-ca.crt\"" Jul 17 02:02:24.004440 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.004412 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979"] Jul 17 02:02:24.145772 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.145734 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-home\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.145772 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.145773 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-dshm\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.146007 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.145796 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-model-cache\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.146007 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.145849 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-tmp-dir\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.146007 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.145882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba58694b-190c-4234-9f3e-086141ca1df5-tls-certs\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.146007 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.145966 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xxlp\" (UniqueName: \"kubernetes.io/projected/ba58694b-190c-4234-9f3e-086141ca1df5-kube-api-access-6xxlp\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.246715 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.246624 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-home\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.246715 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.246663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-dshm\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.246715 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.246685 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-model-cache\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.247000 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.246724 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-tmp-dir\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.247000 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.246871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba58694b-190c-4234-9f3e-086141ca1df5-tls-certs\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.247000 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.246972 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xxlp\" (UniqueName: \"kubernetes.io/projected/ba58694b-190c-4234-9f3e-086141ca1df5-kube-api-access-6xxlp\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.247153 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.247099 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-home\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.247476 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.247450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-tmp-dir\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.247599 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.247486 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-model-cache\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.249398 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.249369 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-dshm\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.250217 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.250193 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba58694b-190c-4234-9f3e-086141ca1df5-tls-certs\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.255000 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.254977 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xxlp\" (UniqueName: \"kubernetes.io/projected/ba58694b-190c-4234-9f3e-086141ca1df5-kube-api-access-6xxlp\") pod \"auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.303395 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.303351 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:24.457620 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:24.457590 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979"] Jul 17 02:02:24.458298 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:02:24.458274 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba58694b_190c_4234_9f3e_086141ca1df5.slice/crio-332c3934648afcb16f99e42c20a4f2ce59024283021e28d6d38852caf3fabc26 WatchSource:0}: Error finding container 332c3934648afcb16f99e42c20a4f2ce59024283021e28d6d38852caf3fabc26: Status 404 returned error can't find the container with id 332c3934648afcb16f99e42c20a4f2ce59024283021e28d6d38852caf3fabc26 Jul 17 02:02:25.158990 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:25.158952 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" event={"ID":"ba58694b-190c-4234-9f3e-086141ca1df5","Type":"ContainerStarted","Data":"c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558"} Jul 17 02:02:25.159452 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:25.158998 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" event={"ID":"ba58694b-190c-4234-9f3e-086141ca1df5","Type":"ContainerStarted","Data":"332c3934648afcb16f99e42c20a4f2ce59024283021e28d6d38852caf3fabc26"} Jul 17 02:02:25.177007 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:25.176956 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" podStartSLOduration=2.17694098 podStartE2EDuration="2.17694098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:02:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:02:25.174483798 +0000 UTC m=+2911.552408206" watchObservedRunningTime="2026-07-17 02:02:25.17694098 +0000 UTC m=+2911.554865388" Jul 17 02:02:34.303818 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:34.303779 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:34.304251 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:34.303833 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:34.315892 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:34.315869 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:02:35.227780 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:02:35.227750 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:03:54.369826 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:03:54.369800 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 02:03:54.381519 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:03:54.381494 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 02:04:53.449443 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:04:53.449352 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979"] Jul 17 02:05:08.472301 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.472261 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" podUID="ba58694b-190c-4234-9f3e-086141ca1df5" containerName="main" containerID="cri-o://c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558" gracePeriod=45 Jul 17 02:05:08.730228 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.730151 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:05:08.878731 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.878699 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xxlp\" (UniqueName: \"kubernetes.io/projected/ba58694b-190c-4234-9f3e-086141ca1df5-kube-api-access-6xxlp\") pod \"ba58694b-190c-4234-9f3e-086141ca1df5\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " Jul 17 02:05:08.878925 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.878849 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-tmp-dir\") pod \"ba58694b-190c-4234-9f3e-086141ca1df5\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " Jul 17 02:05:08.878925 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.878878 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-model-cache\") pod \"ba58694b-190c-4234-9f3e-086141ca1df5\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " Jul 17 02:05:08.878925 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.878905 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba58694b-190c-4234-9f3e-086141ca1df5-tls-certs\") pod \"ba58694b-190c-4234-9f3e-086141ca1df5\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " Jul 17 02:05:08.879093 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.878962 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-home\") pod \"ba58694b-190c-4234-9f3e-086141ca1df5\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " Jul 17 02:05:08.879093 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.878997 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-dshm\") pod \"ba58694b-190c-4234-9f3e-086141ca1df5\" (UID: \"ba58694b-190c-4234-9f3e-086141ca1df5\") " Jul 17 02:05:08.879456 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.879387 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-model-cache" (OuterVolumeSpecName: "model-cache") pod "ba58694b-190c-4234-9f3e-086141ca1df5" (UID: "ba58694b-190c-4234-9f3e-086141ca1df5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:05:08.879760 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.879509 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-home" (OuterVolumeSpecName: "home") pod "ba58694b-190c-4234-9f3e-086141ca1df5" (UID: "ba58694b-190c-4234-9f3e-086141ca1df5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:05:08.879760 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.879543 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ba58694b-190c-4234-9f3e-086141ca1df5" (UID: "ba58694b-190c-4234-9f3e-086141ca1df5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:05:08.881518 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.881495 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba58694b-190c-4234-9f3e-086141ca1df5-kube-api-access-6xxlp" (OuterVolumeSpecName: "kube-api-access-6xxlp") pod "ba58694b-190c-4234-9f3e-086141ca1df5" (UID: "ba58694b-190c-4234-9f3e-086141ca1df5"). InnerVolumeSpecName "kube-api-access-6xxlp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:05:08.881626 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.881551 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-dshm" (OuterVolumeSpecName: "dshm") pod "ba58694b-190c-4234-9f3e-086141ca1df5" (UID: "ba58694b-190c-4234-9f3e-086141ca1df5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:05:08.881836 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.881819 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba58694b-190c-4234-9f3e-086141ca1df5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ba58694b-190c-4234-9f3e-086141ca1df5" (UID: "ba58694b-190c-4234-9f3e-086141ca1df5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 02:05:08.980541 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.980444 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-tmp-dir\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:05:08.980541 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.980475 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-model-cache\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:05:08.980541 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.980487 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba58694b-190c-4234-9f3e-086141ca1df5-tls-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:05:08.980541 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.980494 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-home\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:05:08.980541 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.980503 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ba58694b-190c-4234-9f3e-086141ca1df5-dshm\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:05:08.980541 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:08.980512 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xxlp\" (UniqueName: \"kubernetes.io/projected/ba58694b-190c-4234-9f3e-086141ca1df5-kube-api-access-6xxlp\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:05:09.109270 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:09.109237 2564 generic.go:356] "Generic (PLEG): container finished" podID="ba58694b-190c-4234-9f3e-086141ca1df5" containerID="c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558" exitCode=0 Jul 17 02:05:09.109457 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:09.109280 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" event={"ID":"ba58694b-190c-4234-9f3e-086141ca1df5","Type":"ContainerDied","Data":"c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558"} Jul 17 02:05:09.109457 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:09.109327 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" event={"ID":"ba58694b-190c-4234-9f3e-086141ca1df5","Type":"ContainerDied","Data":"332c3934648afcb16f99e42c20a4f2ce59024283021e28d6d38852caf3fabc26"} Jul 17 02:05:09.109457 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:09.109347 2564 scope.go:117] "RemoveContainer" containerID="c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558" Jul 17 02:05:09.109457 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:09.109289 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979" Jul 17 02:05:09.122670 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:09.122278 2564 scope.go:117] "RemoveContainer" containerID="c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558" Jul 17 02:05:09.122955 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:05:09.122927 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558\": container with ID starting with c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558 not found: ID does not exist" containerID="c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558" Jul 17 02:05:09.123044 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:09.122967 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558"} err="failed to get container status \"c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558\": rpc error: code = NotFound desc = could not find container \"c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558\": container with ID starting with c6d84e981aded4d9eec68296959cf95edd66b426315a1815481bfdf64bcb5558 not found: ID does not exist" Jul 17 02:05:09.136562 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:09.136526 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979"] Jul 17 02:05:09.141504 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:09.141479 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/auth-test-model-fd4ae7-kserve-6c7f74fd67-xl979"] Jul 17 02:05:10.140965 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:10.140930 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba58694b-190c-4234-9f3e-086141ca1df5" path="/var/lib/kubelet/pods/ba58694b-190c-4234-9f3e-086141ca1df5/volumes" Jul 17 02:05:16.177559 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.176900 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5"] Jul 17 02:05:16.180197 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.178836 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba58694b-190c-4234-9f3e-086141ca1df5" containerName="main" Jul 17 02:05:16.180197 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.178864 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba58694b-190c-4234-9f3e-086141ca1df5" containerName="main" Jul 17 02:05:16.180197 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.179060 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba58694b-190c-4234-9f3e-086141ca1df5" containerName="main" Jul 17 02:05:16.185180 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.185139 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.186330 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.186293 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5"] Jul 17 02:05:16.189150 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.189128 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fd4ae7\"/\"kube-root-ca.crt\"" Jul 17 02:05:16.189334 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.189143 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fd4ae7\"/\"default-dockercfg-6bf8t\"" Jul 17 02:05:16.189457 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.189184 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-fd4ae7\"/\"sub-test-model-fd4ae7-kserve-self-signed-certs\"" Jul 17 02:05:16.189582 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.189224 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-fd4ae7\"/\"openshift-service-ca.crt\"" Jul 17 02:05:16.244491 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.244450 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cda2964-da8f-4efe-8c44-d68a646df5f9-tls-certs\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.244666 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.244522 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-dshm\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.244666 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.244642 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqwhx\" (UniqueName: \"kubernetes.io/projected/1cda2964-da8f-4efe-8c44-d68a646df5f9-kube-api-access-cqwhx\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.244742 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.244684 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-model-cache\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.244742 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.244733 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-home\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.244849 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.244836 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-tmp-dir\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.346248 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.346157 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqwhx\" (UniqueName: \"kubernetes.io/projected/1cda2964-da8f-4efe-8c44-d68a646df5f9-kube-api-access-cqwhx\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.346248 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.346256 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-model-cache\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.346481 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.346305 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-home\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.346481 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.346390 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-tmp-dir\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.346481 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.346419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cda2964-da8f-4efe-8c44-d68a646df5f9-tls-certs\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.346481 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.346467 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-dshm\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.346768 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.346744 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-model-cache\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.346815 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.346783 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-home\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.346815 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.346795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-tmp-dir\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.348908 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.348872 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-dshm\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.349141 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.349122 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cda2964-da8f-4efe-8c44-d68a646df5f9-tls-certs\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.354979 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.354955 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqwhx\" (UniqueName: \"kubernetes.io/projected/1cda2964-da8f-4efe-8c44-d68a646df5f9-kube-api-access-cqwhx\") pod \"sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.496486 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.496380 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:16.637012 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:16.636985 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5"] Jul 17 02:05:16.638923 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:05:16.638897 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1cda2964_da8f_4efe_8c44_d68a646df5f9.slice/crio-1d85abb10ca44ff3d905085a0ce9ffd7d9b536cd8316accce65947cfd0023549 WatchSource:0}: Error finding container 1d85abb10ca44ff3d905085a0ce9ffd7d9b536cd8316accce65947cfd0023549: Status 404 returned error can't find the container with id 1d85abb10ca44ff3d905085a0ce9ffd7d9b536cd8316accce65947cfd0023549 Jul 17 02:05:17.155799 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:17.155766 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" event={"ID":"1cda2964-da8f-4efe-8c44-d68a646df5f9","Type":"ContainerStarted","Data":"1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6"} Jul 17 02:05:17.155799 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:17.155813 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" event={"ID":"1cda2964-da8f-4efe-8c44-d68a646df5f9","Type":"ContainerStarted","Data":"1d85abb10ca44ff3d905085a0ce9ffd7d9b536cd8316accce65947cfd0023549"} Jul 17 02:05:17.175027 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:17.174971 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" podStartSLOduration=1.174956064 podStartE2EDuration="1.174956064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:05:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:05:17.173366671 +0000 UTC m=+3083.551291079" watchObservedRunningTime="2026-07-17 02:05:17.174956064 +0000 UTC m=+3083.552880471" Jul 17 02:05:26.497297 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:26.497256 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:26.497775 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:26.497312 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:26.512355 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:26.512313 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:05:27.230537 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:05:27.230501 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:06:20.249547 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:20.249463 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5"] Jul 17 02:06:35.271969 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.271777 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" podUID="1cda2964-da8f-4efe-8c44-d68a646df5f9" containerName="main" containerID="cri-o://1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6" gracePeriod=45 Jul 17 02:06:35.543506 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.543481 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:06:35.614187 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.614130 2564 generic.go:356] "Generic (PLEG): container finished" podID="1cda2964-da8f-4efe-8c44-d68a646df5f9" containerID="1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6" exitCode=0 Jul 17 02:06:35.614363 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.614203 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" Jul 17 02:06:35.614363 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.614215 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" event={"ID":"1cda2964-da8f-4efe-8c44-d68a646df5f9","Type":"ContainerDied","Data":"1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6"} Jul 17 02:06:35.614363 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.614261 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5" event={"ID":"1cda2964-da8f-4efe-8c44-d68a646df5f9","Type":"ContainerDied","Data":"1d85abb10ca44ff3d905085a0ce9ffd7d9b536cd8316accce65947cfd0023549"} Jul 17 02:06:35.614363 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.614281 2564 scope.go:117] "RemoveContainer" containerID="1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6" Jul 17 02:06:35.623438 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.623419 2564 scope.go:117] "RemoveContainer" containerID="1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6" Jul 17 02:06:35.623701 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:06:35.623677 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6\": container with ID starting with 1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6 not found: ID does not exist" containerID="1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6" Jul 17 02:06:35.623762 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.623709 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6"} err="failed to get container status \"1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6\": rpc error: code = NotFound desc = could not find container \"1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6\": container with ID starting with 1c70124dd36b5600f22563530267e43793b3c5067dd3927693e571873e6b66e6 not found: ID does not exist" Jul 17 02:06:35.632464 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.632446 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-dshm\") pod \"1cda2964-da8f-4efe-8c44-d68a646df5f9\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " Jul 17 02:06:35.632543 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.632478 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-home\") pod \"1cda2964-da8f-4efe-8c44-d68a646df5f9\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " Jul 17 02:06:35.632543 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.632499 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-model-cache\") pod \"1cda2964-da8f-4efe-8c44-d68a646df5f9\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " Jul 17 02:06:35.632630 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.632599 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cda2964-da8f-4efe-8c44-d68a646df5f9-tls-certs\") pod \"1cda2964-da8f-4efe-8c44-d68a646df5f9\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " Jul 17 02:06:35.632681 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.632629 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqwhx\" (UniqueName: \"kubernetes.io/projected/1cda2964-da8f-4efe-8c44-d68a646df5f9-kube-api-access-cqwhx\") pod \"1cda2964-da8f-4efe-8c44-d68a646df5f9\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " Jul 17 02:06:35.632726 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.632681 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-tmp-dir\") pod \"1cda2964-da8f-4efe-8c44-d68a646df5f9\" (UID: \"1cda2964-da8f-4efe-8c44-d68a646df5f9\") " Jul 17 02:06:35.632777 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.632712 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-home" (OuterVolumeSpecName: "home") pod "1cda2964-da8f-4efe-8c44-d68a646df5f9" (UID: "1cda2964-da8f-4efe-8c44-d68a646df5f9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:06:35.632829 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.632779 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-model-cache" (OuterVolumeSpecName: "model-cache") pod "1cda2964-da8f-4efe-8c44-d68a646df5f9" (UID: "1cda2964-da8f-4efe-8c44-d68a646df5f9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:06:35.632980 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.632958 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1cda2964-da8f-4efe-8c44-d68a646df5f9" (UID: "1cda2964-da8f-4efe-8c44-d68a646df5f9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:06:35.633078 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.633063 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-tmp-dir\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:06:35.633134 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.633080 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-home\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:06:35.633134 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.633089 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-model-cache\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:06:35.634752 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.634732 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-dshm" (OuterVolumeSpecName: "dshm") pod "1cda2964-da8f-4efe-8c44-d68a646df5f9" (UID: "1cda2964-da8f-4efe-8c44-d68a646df5f9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:06:35.634847 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.634803 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cda2964-da8f-4efe-8c44-d68a646df5f9-kube-api-access-cqwhx" (OuterVolumeSpecName: "kube-api-access-cqwhx") pod "1cda2964-da8f-4efe-8c44-d68a646df5f9" (UID: "1cda2964-da8f-4efe-8c44-d68a646df5f9"). InnerVolumeSpecName "kube-api-access-cqwhx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:06:35.634847 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.634826 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cda2964-da8f-4efe-8c44-d68a646df5f9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1cda2964-da8f-4efe-8c44-d68a646df5f9" (UID: "1cda2964-da8f-4efe-8c44-d68a646df5f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 02:06:35.734421 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.734379 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1cda2964-da8f-4efe-8c44-d68a646df5f9-dshm\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:06:35.734421 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.734416 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1cda2964-da8f-4efe-8c44-d68a646df5f9-tls-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:06:35.734421 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.734431 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqwhx\" (UniqueName: \"kubernetes.io/projected/1cda2964-da8f-4efe-8c44-d68a646df5f9-kube-api-access-cqwhx\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:06:35.937067 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.937022 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5"] Jul 17 02:06:35.940264 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:35.940237 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-fd4ae7/sub-test-model-fd4ae7-kserve-8c66868f7-sf6z5"] Jul 17 02:06:36.133841 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:06:36.133807 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cda2964-da8f-4efe-8c44-d68a646df5f9" path="/var/lib/kubelet/pods/1cda2964-da8f-4efe-8c44-d68a646df5f9/volumes" Jul 17 02:07:35.694260 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.694224 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7502-139828471601840"] Jul 17 02:07:35.694703 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.694667 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cda2964-da8f-4efe-8c44-d68a646df5f9" containerName="main" Jul 17 02:07:35.694703 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.694680 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cda2964-da8f-4efe-8c44-d68a646df5f9" containerName="main" Jul 17 02:07:35.694779 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.694750 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cda2964-da8f-4efe-8c44-d68a646df5f9" containerName="main" Jul 17 02:07:35.697794 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.697778 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471601840" Jul 17 02:07:35.700952 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.700925 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gt8p5\"" Jul 17 02:07:35.706255 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.706231 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471601840"] Jul 17 02:07:35.831672 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.831635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6r4d\" (UniqueName: \"kubernetes.io/projected/e0316840-1b65-40d3-8c85-6964cbf76433-kube-api-access-s6r4d\") pod \"test-curl-7502-139828471601840\" (UID: \"e0316840-1b65-40d3-8c85-6964cbf76433\") " pod="opendatahub/test-curl-7502-139828471601840" Jul 17 02:07:35.933173 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.933129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6r4d\" (UniqueName: \"kubernetes.io/projected/e0316840-1b65-40d3-8c85-6964cbf76433-kube-api-access-s6r4d\") pod \"test-curl-7502-139828471601840\" (UID: \"e0316840-1b65-40d3-8c85-6964cbf76433\") " pod="opendatahub/test-curl-7502-139828471601840" Jul 17 02:07:35.941848 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:35.941822 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6r4d\" (UniqueName: \"kubernetes.io/projected/e0316840-1b65-40d3-8c85-6964cbf76433-kube-api-access-s6r4d\") pod \"test-curl-7502-139828471601840\" (UID: \"e0316840-1b65-40d3-8c85-6964cbf76433\") " pod="opendatahub/test-curl-7502-139828471601840" Jul 17 02:07:36.010605 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:36.010520 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471601840" Jul 17 02:07:36.149432 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:36.149402 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471601840"] Jul 17 02:07:36.150983 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:07:36.150955 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0316840_1b65_40d3_8c85_6964cbf76433.slice/crio-d3c1dfc491e3f2f3607a247278799afd16da6298faa0711db3f3861efa1e3f4c WatchSource:0}: Error finding container d3c1dfc491e3f2f3607a247278799afd16da6298faa0711db3f3861efa1e3f4c: Status 404 returned error can't find the container with id d3c1dfc491e3f2f3607a247278799afd16da6298faa0711db3f3861efa1e3f4c Jul 17 02:07:36.152334 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:36.152309 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 02:07:36.983605 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:36.983569 2564 generic.go:356] "Generic (PLEG): container finished" podID="e0316840-1b65-40d3-8c85-6964cbf76433" containerID="68baa8628bf5ab234dd5e08bbfa1bd8c531069979ddc65af30a21982b3dd4b8a" exitCode=0 Jul 17 02:07:36.984042 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:36.983615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471601840" event={"ID":"e0316840-1b65-40d3-8c85-6964cbf76433","Type":"ContainerDied","Data":"68baa8628bf5ab234dd5e08bbfa1bd8c531069979ddc65af30a21982b3dd4b8a"} Jul 17 02:07:36.984042 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:36.983679 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471601840" event={"ID":"e0316840-1b65-40d3-8c85-6964cbf76433","Type":"ContainerStarted","Data":"d3c1dfc491e3f2f3607a247278799afd16da6298faa0711db3f3861efa1e3f4c"} Jul 17 02:07:38.142815 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.142784 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471601840" Jul 17 02:07:38.177958 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.177923 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7502-139828471601840_e0316840-1b65-40d3-8c85-6964cbf76433/test-curl-7502-139828471601840/0.log" Jul 17 02:07:38.201985 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.201944 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7502-139828471601840"] Jul 17 02:07:38.204489 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.204466 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7502-139828471601840"] Jul 17 02:07:38.257666 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.257578 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6r4d\" (UniqueName: \"kubernetes.io/projected/e0316840-1b65-40d3-8c85-6964cbf76433-kube-api-access-s6r4d\") pod \"e0316840-1b65-40d3-8c85-6964cbf76433\" (UID: \"e0316840-1b65-40d3-8c85-6964cbf76433\") " Jul 17 02:07:38.259770 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.259738 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0316840-1b65-40d3-8c85-6964cbf76433-kube-api-access-s6r4d" (OuterVolumeSpecName: "kube-api-access-s6r4d") pod "e0316840-1b65-40d3-8c85-6964cbf76433" (UID: "e0316840-1b65-40d3-8c85-6964cbf76433"). InnerVolumeSpecName "kube-api-access-s6r4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:07:38.324571 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.324535 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7502-139828471419696"] Jul 17 02:07:38.324974 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.324960 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0316840-1b65-40d3-8c85-6964cbf76433" containerName="test-curl-7502-139828471601840" Jul 17 02:07:38.325027 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.324975 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0316840-1b65-40d3-8c85-6964cbf76433" containerName="test-curl-7502-139828471601840" Jul 17 02:07:38.325062 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.325052 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0316840-1b65-40d3-8c85-6964cbf76433" containerName="test-curl-7502-139828471601840" Jul 17 02:07:38.328074 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.328053 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471419696" Jul 17 02:07:38.334062 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.334035 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471419696"] Jul 17 02:07:38.358568 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.358540 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6r4d\" (UniqueName: \"kubernetes.io/projected/e0316840-1b65-40d3-8c85-6964cbf76433-kube-api-access-s6r4d\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:07:38.459970 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.459935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmnkk\" (UniqueName: \"kubernetes.io/projected/e606dc67-314e-4548-b401-9ad6dc400e52-kube-api-access-bmnkk\") pod \"test-curl-7502-139828471419696\" (UID: \"e606dc67-314e-4548-b401-9ad6dc400e52\") " pod="opendatahub/test-curl-7502-139828471419696" Jul 17 02:07:38.561321 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.561216 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmnkk\" (UniqueName: \"kubernetes.io/projected/e606dc67-314e-4548-b401-9ad6dc400e52-kube-api-access-bmnkk\") pod \"test-curl-7502-139828471419696\" (UID: \"e606dc67-314e-4548-b401-9ad6dc400e52\") " pod="opendatahub/test-curl-7502-139828471419696" Jul 17 02:07:38.570559 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.570532 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmnkk\" (UniqueName: \"kubernetes.io/projected/e606dc67-314e-4548-b401-9ad6dc400e52-kube-api-access-bmnkk\") pod \"test-curl-7502-139828471419696\" (UID: \"e606dc67-314e-4548-b401-9ad6dc400e52\") " pod="opendatahub/test-curl-7502-139828471419696" Jul 17 02:07:38.639533 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.639497 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471419696" Jul 17 02:07:38.784433 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.784408 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471419696"] Jul 17 02:07:38.785902 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:07:38.785875 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode606dc67_314e_4548_b401_9ad6dc400e52.slice/crio-d13c96b0ec53b50d39025fb25724e8b6cb7e16f228c2f6dab9b735f501d825c1 WatchSource:0}: Error finding container d13c96b0ec53b50d39025fb25724e8b6cb7e16f228c2f6dab9b735f501d825c1: Status 404 returned error can't find the container with id d13c96b0ec53b50d39025fb25724e8b6cb7e16f228c2f6dab9b735f501d825c1 Jul 17 02:07:39.000021 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:38.999978 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471419696" event={"ID":"e606dc67-314e-4548-b401-9ad6dc400e52","Type":"ContainerStarted","Data":"d13c96b0ec53b50d39025fb25724e8b6cb7e16f228c2f6dab9b735f501d825c1"} Jul 17 02:07:39.001299 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:39.001278 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471601840" Jul 17 02:07:39.001404 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:39.001286 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3c1dfc491e3f2f3607a247278799afd16da6298faa0711db3f3861efa1e3f4c" Jul 17 02:07:39.012736 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:07:39.012706 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7502-139828471601840\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="e0316840-1b65-40d3-8c85-6964cbf76433" pod="opendatahub/test-curl-7502-139828471601840" Jul 17 02:07:40.008655 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:40.008612 2564 generic.go:356] "Generic (PLEG): container finished" podID="e606dc67-314e-4548-b401-9ad6dc400e52" containerID="d3fe19a17ecd53327c62246b2c7d6833f028bb07675e53402c7fc1cc930b68b2" exitCode=0 Jul 17 02:07:40.009067 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:40.008699 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471419696" event={"ID":"e606dc67-314e-4548-b401-9ad6dc400e52","Type":"ContainerDied","Data":"d3fe19a17ecd53327c62246b2c7d6833f028bb07675e53402c7fc1cc930b68b2"} Jul 17 02:07:40.011076 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:07:40.011050 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7502-139828471601840\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="e0316840-1b65-40d3-8c85-6964cbf76433" pod="opendatahub/test-curl-7502-139828471601840" Jul 17 02:07:40.140534 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:40.136710 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0316840-1b65-40d3-8c85-6964cbf76433" path="/var/lib/kubelet/pods/e0316840-1b65-40d3-8c85-6964cbf76433/volumes" Jul 17 02:07:41.162438 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.162413 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471419696" Jul 17 02:07:41.182835 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.182799 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7502-139828471419696_e606dc67-314e-4548-b401-9ad6dc400e52/test-curl-7502-139828471419696/0.log" Jul 17 02:07:41.211605 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.211569 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7502-139828471419696"] Jul 17 02:07:41.213962 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.213943 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7502-139828471419696"] Jul 17 02:07:41.288596 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.288507 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmnkk\" (UniqueName: \"kubernetes.io/projected/e606dc67-314e-4548-b401-9ad6dc400e52-kube-api-access-bmnkk\") pod \"e606dc67-314e-4548-b401-9ad6dc400e52\" (UID: \"e606dc67-314e-4548-b401-9ad6dc400e52\") " Jul 17 02:07:41.290676 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.290651 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e606dc67-314e-4548-b401-9ad6dc400e52-kube-api-access-bmnkk" (OuterVolumeSpecName: "kube-api-access-bmnkk") pod "e606dc67-314e-4548-b401-9ad6dc400e52" (UID: "e606dc67-314e-4548-b401-9ad6dc400e52"). InnerVolumeSpecName "kube-api-access-bmnkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:07:41.383924 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.383885 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:41.384347 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.384332 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e606dc67-314e-4548-b401-9ad6dc400e52" containerName="test-curl-7502-139828471419696" Jul 17 02:07:41.384396 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.384348 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e606dc67-314e-4548-b401-9ad6dc400e52" containerName="test-curl-7502-139828471419696" Jul 17 02:07:41.384443 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.384430 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e606dc67-314e-4548-b401-9ad6dc400e52" containerName="test-curl-7502-139828471419696" Jul 17 02:07:41.387856 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.387827 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:41.389591 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.389552 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmnkk\" (UniqueName: \"kubernetes.io/projected/e606dc67-314e-4548-b401-9ad6dc400e52-kube-api-access-bmnkk\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:07:41.391308 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.391274 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:41.490406 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.490374 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zczq6\" (UniqueName: \"kubernetes.io/projected/a136e784-6e35-4eef-80a3-7403aee4301d-kube-api-access-zczq6\") pod \"test-curl-7502-139828471362880\" (UID: \"a136e784-6e35-4eef-80a3-7403aee4301d\") " pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:41.591373 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.591274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zczq6\" (UniqueName: \"kubernetes.io/projected/a136e784-6e35-4eef-80a3-7403aee4301d-kube-api-access-zczq6\") pod \"test-curl-7502-139828471362880\" (UID: \"a136e784-6e35-4eef-80a3-7403aee4301d\") " pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:41.599864 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.599834 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zczq6\" (UniqueName: \"kubernetes.io/projected/a136e784-6e35-4eef-80a3-7403aee4301d-kube-api-access-zczq6\") pod \"test-curl-7502-139828471362880\" (UID: \"a136e784-6e35-4eef-80a3-7403aee4301d\") " pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:41.700446 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.700411 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:41.843884 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:41.843853 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:41.845139 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:07:41.845114 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda136e784_6e35_4eef_80a3_7403aee4301d.slice/crio-86d68cdfe44ab50e1ef11f7bd4b310718eaf950cb2f232584e49515b88bbd1fa WatchSource:0}: Error finding container 86d68cdfe44ab50e1ef11f7bd4b310718eaf950cb2f232584e49515b88bbd1fa: Status 404 returned error can't find the container with id 86d68cdfe44ab50e1ef11f7bd4b310718eaf950cb2f232584e49515b88bbd1fa Jul 17 02:07:42.021826 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:42.021788 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471362880" event={"ID":"a136e784-6e35-4eef-80a3-7403aee4301d","Type":"ContainerStarted","Data":"86d68cdfe44ab50e1ef11f7bd4b310718eaf950cb2f232584e49515b88bbd1fa"} Jul 17 02:07:42.023082 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:42.023059 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471419696" Jul 17 02:07:42.023233 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:42.023059 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d13c96b0ec53b50d39025fb25724e8b6cb7e16f228c2f6dab9b735f501d825c1" Jul 17 02:07:42.034976 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:07:42.034947 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7502-139828471419696\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="e606dc67-314e-4548-b401-9ad6dc400e52" pod="opendatahub/test-curl-7502-139828471419696" Jul 17 02:07:42.136523 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:42.136491 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e606dc67-314e-4548-b401-9ad6dc400e52" path="/var/lib/kubelet/pods/e606dc67-314e-4548-b401-9ad6dc400e52/volumes" Jul 17 02:07:43.029917 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:43.029879 2564 generic.go:356] "Generic (PLEG): container finished" podID="a136e784-6e35-4eef-80a3-7403aee4301d" containerID="2eaeea35466ca02045c341a978a5addee76eefa10221f74dae96f0adb901e484" exitCode=0 Jul 17 02:07:43.030405 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:43.029959 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471362880" event={"ID":"a136e784-6e35-4eef-80a3-7403aee4301d","Type":"ContainerDied","Data":"2eaeea35466ca02045c341a978a5addee76eefa10221f74dae96f0adb901e484"} Jul 17 02:07:44.182974 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.182949 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:44.202034 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.201985 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7502-139828471362880_a136e784-6e35-4eef-80a3-7403aee4301d/test-curl-7502-139828471362880/0.log" Jul 17 02:07:44.227970 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.227663 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:44.235678 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.235636 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:44.319476 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.319379 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zczq6\" (UniqueName: \"kubernetes.io/projected/a136e784-6e35-4eef-80a3-7403aee4301d-kube-api-access-zczq6\") pod \"a136e784-6e35-4eef-80a3-7403aee4301d\" (UID: \"a136e784-6e35-4eef-80a3-7403aee4301d\") " Jul 17 02:07:44.321701 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.321669 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a136e784-6e35-4eef-80a3-7403aee4301d-kube-api-access-zczq6" (OuterVolumeSpecName: "kube-api-access-zczq6") pod "a136e784-6e35-4eef-80a3-7403aee4301d" (UID: "a136e784-6e35-4eef-80a3-7403aee4301d"). InnerVolumeSpecName "kube-api-access-zczq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:07:44.346487 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.346447 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:44.346936 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.346920 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a136e784-6e35-4eef-80a3-7403aee4301d" containerName="test-curl-7502-139828471362880" Jul 17 02:07:44.346936 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.346936 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a136e784-6e35-4eef-80a3-7403aee4301d" containerName="test-curl-7502-139828471362880" Jul 17 02:07:44.347025 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.346997 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a136e784-6e35-4eef-80a3-7403aee4301d" containerName="test-curl-7502-139828471362880" Jul 17 02:07:44.350085 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.350071 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:44.356261 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.356232 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:44.420664 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.420628 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zczq6\" (UniqueName: \"kubernetes.io/projected/a136e784-6e35-4eef-80a3-7403aee4301d-kube-api-access-zczq6\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:07:44.521784 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.521753 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v2q4\" (UniqueName: \"kubernetes.io/projected/3eaf989a-bcfb-4058-8b3c-207db4503374-kube-api-access-7v2q4\") pod \"test-curl-7502-139828471362880\" (UID: \"3eaf989a-bcfb-4058-8b3c-207db4503374\") " pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:44.622987 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.622898 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v2q4\" (UniqueName: \"kubernetes.io/projected/3eaf989a-bcfb-4058-8b3c-207db4503374-kube-api-access-7v2q4\") pod \"test-curl-7502-139828471362880\" (UID: \"3eaf989a-bcfb-4058-8b3c-207db4503374\") " pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:44.631157 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.631118 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v2q4\" (UniqueName: \"kubernetes.io/projected/3eaf989a-bcfb-4058-8b3c-207db4503374-kube-api-access-7v2q4\") pod \"test-curl-7502-139828471362880\" (UID: \"3eaf989a-bcfb-4058-8b3c-207db4503374\") " pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:44.660771 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.660738 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:44.803823 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:44.803786 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:44.805054 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:07:44.805031 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3eaf989a_bcfb_4058_8b3c_207db4503374.slice/crio-318ecb08ec45c2188b2346685de1c12dcc4943405fa44a8121e4a49af0db292d WatchSource:0}: Error finding container 318ecb08ec45c2188b2346685de1c12dcc4943405fa44a8121e4a49af0db292d: Status 404 returned error can't find the container with id 318ecb08ec45c2188b2346685de1c12dcc4943405fa44a8121e4a49af0db292d Jul 17 02:07:45.043203 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:45.043137 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471362880" event={"ID":"3eaf989a-bcfb-4058-8b3c-207db4503374","Type":"ContainerStarted","Data":"318ecb08ec45c2188b2346685de1c12dcc4943405fa44a8121e4a49af0db292d"} Jul 17 02:07:45.044398 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:45.044378 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:45.044398 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:45.044394 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="86d68cdfe44ab50e1ef11f7bd4b310718eaf950cb2f232584e49515b88bbd1fa" Jul 17 02:07:45.056124 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:45.056090 2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7502-139828471362880" oldPodUID="a136e784-6e35-4eef-80a3-7403aee4301d" podUID="3eaf989a-bcfb-4058-8b3c-207db4503374" Jul 17 02:07:46.051431 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:46.051400 2564 generic.go:356] "Generic (PLEG): container finished" podID="3eaf989a-bcfb-4058-8b3c-207db4503374" containerID="3c4a790cc373ab908e7b3d9eb3bccdf5f94004823e2c30166133b8a35fe7001e" exitCode=0 Jul 17 02:07:46.051831 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:46.051492 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471362880" event={"ID":"3eaf989a-bcfb-4058-8b3c-207db4503374","Type":"ContainerDied","Data":"3c4a790cc373ab908e7b3d9eb3bccdf5f94004823e2c30166133b8a35fe7001e"} Jul 17 02:07:46.134823 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:46.134792 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a136e784-6e35-4eef-80a3-7403aee4301d" path="/var/lib/kubelet/pods/a136e784-6e35-4eef-80a3-7403aee4301d/volumes" Jul 17 02:07:47.196748 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.196725 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:47.218345 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.218239 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7502-139828471362880_3eaf989a-bcfb-4058-8b3c-207db4503374/test-curl-7502-139828471362880/0.log" Jul 17 02:07:47.244231 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.244198 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:47.247813 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.247782 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7502-139828471362880"] Jul 17 02:07:47.348810 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.348724 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v2q4\" (UniqueName: \"kubernetes.io/projected/3eaf989a-bcfb-4058-8b3c-207db4503374-kube-api-access-7v2q4\") pod \"3eaf989a-bcfb-4058-8b3c-207db4503374\" (UID: \"3eaf989a-bcfb-4058-8b3c-207db4503374\") " Jul 17 02:07:47.350929 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.350907 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eaf989a-bcfb-4058-8b3c-207db4503374-kube-api-access-7v2q4" (OuterVolumeSpecName: "kube-api-access-7v2q4") pod "3eaf989a-bcfb-4058-8b3c-207db4503374" (UID: "3eaf989a-bcfb-4058-8b3c-207db4503374"). InnerVolumeSpecName "kube-api-access-7v2q4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:07:47.365806 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.365776 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:47.366237 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.366220 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eaf989a-bcfb-4058-8b3c-207db4503374" containerName="test-curl-7502-139828471362880" Jul 17 02:07:47.366291 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.366241 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eaf989a-bcfb-4058-8b3c-207db4503374" containerName="test-curl-7502-139828471362880" Jul 17 02:07:47.366349 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.366339 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eaf989a-bcfb-4058-8b3c-207db4503374" containerName="test-curl-7502-139828471362880" Jul 17 02:07:47.369320 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.369305 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:47.376108 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.375769 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:47.449535 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.449496 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v2q4\" (UniqueName: \"kubernetes.io/projected/3eaf989a-bcfb-4058-8b3c-207db4503374-kube-api-access-7v2q4\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:07:47.550720 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.550677 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p87zv\" (UniqueName: \"kubernetes.io/projected/d965b259-7bd9-4874-b50c-02698ec9d3cf-kube-api-access-p87zv\") pod \"test-curl-7502-139828471363888\" (UID: \"d965b259-7bd9-4874-b50c-02698ec9d3cf\") " pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:47.652369 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.652339 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p87zv\" (UniqueName: \"kubernetes.io/projected/d965b259-7bd9-4874-b50c-02698ec9d3cf-kube-api-access-p87zv\") pod \"test-curl-7502-139828471363888\" (UID: \"d965b259-7bd9-4874-b50c-02698ec9d3cf\") " pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:47.660406 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.660370 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p87zv\" (UniqueName: \"kubernetes.io/projected/d965b259-7bd9-4874-b50c-02698ec9d3cf-kube-api-access-p87zv\") pod \"test-curl-7502-139828471363888\" (UID: \"d965b259-7bd9-4874-b50c-02698ec9d3cf\") " pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:47.681262 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.681220 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:47.826752 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:47.826724 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:47.828304 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:07:47.828281 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd965b259_7bd9_4874_b50c_02698ec9d3cf.slice/crio-1a2d4febd5b5785f966599ae2918baa34bd9dfab1d1bd5239124aa169ad924f8 WatchSource:0}: Error finding container 1a2d4febd5b5785f966599ae2918baa34bd9dfab1d1bd5239124aa169ad924f8: Status 404 returned error can't find the container with id 1a2d4febd5b5785f966599ae2918baa34bd9dfab1d1bd5239124aa169ad924f8 Jul 17 02:07:48.064558 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:48.064459 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471363888" event={"ID":"d965b259-7bd9-4874-b50c-02698ec9d3cf","Type":"ContainerStarted","Data":"1a2d4febd5b5785f966599ae2918baa34bd9dfab1d1bd5239124aa169ad924f8"} Jul 17 02:07:48.065733 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:48.065710 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:48.065879 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:48.065754 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="318ecb08ec45c2188b2346685de1c12dcc4943405fa44a8121e4a49af0db292d" Jul 17 02:07:48.076298 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:07:48.076268 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7502-139828471362880\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="3eaf989a-bcfb-4058-8b3c-207db4503374" pod="opendatahub/test-curl-7502-139828471362880" Jul 17 02:07:48.134437 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:48.134401 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eaf989a-bcfb-4058-8b3c-207db4503374" path="/var/lib/kubelet/pods/3eaf989a-bcfb-4058-8b3c-207db4503374/volumes" Jul 17 02:07:49.073296 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:49.073257 2564 generic.go:356] "Generic (PLEG): container finished" podID="d965b259-7bd9-4874-b50c-02698ec9d3cf" containerID="67829b0c6536e57e231814189cf18ec54fbece6d7e19aeb09a31ae790cd8ca0f" exitCode=0 Jul 17 02:07:49.073743 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:49.073340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471363888" event={"ID":"d965b259-7bd9-4874-b50c-02698ec9d3cf","Type":"ContainerDied","Data":"67829b0c6536e57e231814189cf18ec54fbece6d7e19aeb09a31ae790cd8ca0f"} Jul 17 02:07:50.225667 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.225641 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:50.244401 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.244374 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7502-139828471363888_d965b259-7bd9-4874-b50c-02698ec9d3cf/test-curl-7502-139828471363888/0.log" Jul 17 02:07:50.270325 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.270295 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:50.274960 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.274925 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:50.282453 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.282431 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p87zv\" (UniqueName: \"kubernetes.io/projected/d965b259-7bd9-4874-b50c-02698ec9d3cf-kube-api-access-p87zv\") pod \"d965b259-7bd9-4874-b50c-02698ec9d3cf\" (UID: \"d965b259-7bd9-4874-b50c-02698ec9d3cf\") " Jul 17 02:07:50.284771 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.284748 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d965b259-7bd9-4874-b50c-02698ec9d3cf-kube-api-access-p87zv" (OuterVolumeSpecName: "kube-api-access-p87zv") pod "d965b259-7bd9-4874-b50c-02698ec9d3cf" (UID: "d965b259-7bd9-4874-b50c-02698ec9d3cf"). InnerVolumeSpecName "kube-api-access-p87zv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:07:50.384025 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.383939 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p87zv\" (UniqueName: \"kubernetes.io/projected/d965b259-7bd9-4874-b50c-02698ec9d3cf-kube-api-access-p87zv\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:07:50.399346 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.399315 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:50.399768 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.399753 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d965b259-7bd9-4874-b50c-02698ec9d3cf" containerName="test-curl-7502-139828471363888" Jul 17 02:07:50.399823 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.399770 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d965b259-7bd9-4874-b50c-02698ec9d3cf" containerName="test-curl-7502-139828471363888" Jul 17 02:07:50.399862 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.399835 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d965b259-7bd9-4874-b50c-02698ec9d3cf" containerName="test-curl-7502-139828471363888" Jul 17 02:07:50.402945 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.402930 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:50.408986 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.408960 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:50.484440 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.484405 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hhtm\" (UniqueName: \"kubernetes.io/projected/30968c18-07d5-44bf-9121-bcfc763670fa-kube-api-access-6hhtm\") pod \"test-curl-7502-139828471363888\" (UID: \"30968c18-07d5-44bf-9121-bcfc763670fa\") " pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:50.585353 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.585320 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hhtm\" (UniqueName: \"kubernetes.io/projected/30968c18-07d5-44bf-9121-bcfc763670fa-kube-api-access-6hhtm\") pod \"test-curl-7502-139828471363888\" (UID: \"30968c18-07d5-44bf-9121-bcfc763670fa\") " pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:50.593560 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.593518 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hhtm\" (UniqueName: \"kubernetes.io/projected/30968c18-07d5-44bf-9121-bcfc763670fa-kube-api-access-6hhtm\") pod \"test-curl-7502-139828471363888\" (UID: \"30968c18-07d5-44bf-9121-bcfc763670fa\") " pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:50.715390 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.715358 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:50.860555 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:50.860521 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:50.862318 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:07:50.862292 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30968c18_07d5_44bf_9121_bcfc763670fa.slice/crio-6b82c81ba3a4a16718a22f0d838acff520edadd55c0bd7c9daa5ee4c4f0784c6 WatchSource:0}: Error finding container 6b82c81ba3a4a16718a22f0d838acff520edadd55c0bd7c9daa5ee4c4f0784c6: Status 404 returned error can't find the container with id 6b82c81ba3a4a16718a22f0d838acff520edadd55c0bd7c9daa5ee4c4f0784c6 Jul 17 02:07:51.085556 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:51.085467 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:51.085556 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:51.085498 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a2d4febd5b5785f966599ae2918baa34bd9dfab1d1bd5239124aa169ad924f8" Jul 17 02:07:51.086707 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:51.086669 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471363888" event={"ID":"30968c18-07d5-44bf-9121-bcfc763670fa","Type":"ContainerStarted","Data":"6b82c81ba3a4a16718a22f0d838acff520edadd55c0bd7c9daa5ee4c4f0784c6"} Jul 17 02:07:51.097267 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:51.097240 2564 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7502-139828471363888" oldPodUID="d965b259-7bd9-4874-b50c-02698ec9d3cf" podUID="30968c18-07d5-44bf-9121-bcfc763670fa" Jul 17 02:07:52.093915 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:52.093883 2564 generic.go:356] "Generic (PLEG): container finished" podID="30968c18-07d5-44bf-9121-bcfc763670fa" containerID="05b0a20398b7ec613797f753d7567e74fea8361e88c58994a37c4e92998d0cc5" exitCode=0 Jul 17 02:07:52.094321 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:52.093972 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7502-139828471363888" event={"ID":"30968c18-07d5-44bf-9121-bcfc763670fa","Type":"ContainerDied","Data":"05b0a20398b7ec613797f753d7567e74fea8361e88c58994a37c4e92998d0cc5"} Jul 17 02:07:52.134268 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:52.134238 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d965b259-7bd9-4874-b50c-02698ec9d3cf" path="/var/lib/kubelet/pods/d965b259-7bd9-4874-b50c-02698ec9d3cf/volumes" Jul 17 02:07:53.241808 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:53.241779 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:53.263383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:53.263265 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7502-139828471363888_30968c18-07d5-44bf-9121-bcfc763670fa/test-curl-7502-139828471363888/0.log" Jul 17 02:07:53.288890 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:53.288853 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:53.297609 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:53.292426 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7502-139828471363888"] Jul 17 02:07:53.308057 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:53.308028 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hhtm\" (UniqueName: \"kubernetes.io/projected/30968c18-07d5-44bf-9121-bcfc763670fa-kube-api-access-6hhtm\") pod \"30968c18-07d5-44bf-9121-bcfc763670fa\" (UID: \"30968c18-07d5-44bf-9121-bcfc763670fa\") " Jul 17 02:07:53.310313 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:53.310286 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30968c18-07d5-44bf-9121-bcfc763670fa-kube-api-access-6hhtm" (OuterVolumeSpecName: "kube-api-access-6hhtm") pod "30968c18-07d5-44bf-9121-bcfc763670fa" (UID: "30968c18-07d5-44bf-9121-bcfc763670fa"). InnerVolumeSpecName "kube-api-access-6hhtm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:07:53.409840 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:53.409803 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hhtm\" (UniqueName: \"kubernetes.io/projected/30968c18-07d5-44bf-9121-bcfc763670fa-kube-api-access-6hhtm\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:07:54.109583 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:54.109553 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b82c81ba3a4a16718a22f0d838acff520edadd55c0bd7c9daa5ee4c4f0784c6" Jul 17 02:07:54.109583 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:54.109580 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:54.119741 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:07:54.119702 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7502-139828471363888\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="30968c18-07d5-44bf-9121-bcfc763670fa" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:54.134744 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:07:54.134687 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7502-139828471363888\" is forbidden: User \"system:node:ip-10-0-136-58.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-58.ec2.internal' and this object" podUID="30968c18-07d5-44bf-9121-bcfc763670fa" pod="opendatahub/test-curl-7502-139828471363888" Jul 17 02:07:54.136693 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:07:54.136669 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30968c18-07d5-44bf-9121-bcfc763670fa" path="/var/lib/kubelet/pods/30968c18-07d5-44bf-9121-bcfc763670fa/volumes" Jul 17 02:08:19.244880 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:19.244844 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw"] Jul 17 02:08:19.245376 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:19.245183 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" containerID="cri-o://cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942" gracePeriod=30 Jul 17 02:08:24.506256 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.506227 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:08:24.621537 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621449 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-credential-socket\") pod \"ff59dab7-2302-49dc-a390-661fc2568b3f\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " Jul 17 02:08:24.621537 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621489 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-envoy\") pod \"ff59dab7-2302-49dc-a390-661fc2568b3f\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " Jul 17 02:08:24.621537 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621511 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-podinfo\") pod \"ff59dab7-2302-49dc-a390-661fc2568b3f\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " Jul 17 02:08:24.621537 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621538 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-certs\") pod \"ff59dab7-2302-49dc-a390-661fc2568b3f\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " Jul 17 02:08:24.621878 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621568 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-socket\") pod \"ff59dab7-2302-49dc-a390-661fc2568b3f\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " Jul 17 02:08:24.621878 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621618 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-token\") pod \"ff59dab7-2302-49dc-a390-661fc2568b3f\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " Jul 17 02:08:24.621878 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621680 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff59dab7-2302-49dc-a390-661fc2568b3f-istiod-ca-cert\") pod \"ff59dab7-2302-49dc-a390-661fc2568b3f\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " Jul 17 02:08:24.621878 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621724 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjb5f\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-kube-api-access-bjb5f\") pod \"ff59dab7-2302-49dc-a390-661fc2568b3f\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " Jul 17 02:08:24.621878 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621752 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-data\") pod \"ff59dab7-2302-49dc-a390-661fc2568b3f\" (UID: \"ff59dab7-2302-49dc-a390-661fc2568b3f\") " Jul 17 02:08:24.621878 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.621809 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff59dab7-2302-49dc-a390-661fc2568b3f" (UID: "ff59dab7-2302-49dc-a390-661fc2568b3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:08:24.622185 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.622114 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-credential-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:08:24.622185 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.622127 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff59dab7-2302-49dc-a390-661fc2568b3f" (UID: "ff59dab7-2302-49dc-a390-661fc2568b3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:08:24.622305 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.622290 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff59dab7-2302-49dc-a390-661fc2568b3f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff59dab7-2302-49dc-a390-661fc2568b3f" (UID: "ff59dab7-2302-49dc-a390-661fc2568b3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 02:08:24.622580 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.622541 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff59dab7-2302-49dc-a390-661fc2568b3f" (UID: "ff59dab7-2302-49dc-a390-661fc2568b3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:08:24.622721 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.622627 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff59dab7-2302-49dc-a390-661fc2568b3f" (UID: "ff59dab7-2302-49dc-a390-661fc2568b3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:08:24.624620 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.624590 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff59dab7-2302-49dc-a390-661fc2568b3f" (UID: "ff59dab7-2302-49dc-a390-661fc2568b3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:08:24.624805 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.624785 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff59dab7-2302-49dc-a390-661fc2568b3f" (UID: "ff59dab7-2302-49dc-a390-661fc2568b3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 02:08:24.624916 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.624850 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff59dab7-2302-49dc-a390-661fc2568b3f" (UID: "ff59dab7-2302-49dc-a390-661fc2568b3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 02:08:24.624984 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.624933 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-kube-api-access-bjb5f" (OuterVolumeSpecName: "kube-api-access-bjb5f") pod "ff59dab7-2302-49dc-a390-661fc2568b3f" (UID: "ff59dab7-2302-49dc-a390-661fc2568b3f"). InnerVolumeSpecName "kube-api-access-bjb5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 02:08:24.723223 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.723155 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-envoy\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:08:24.723223 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.723219 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-podinfo\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:08:24.723223 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.723230 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-certs\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:08:24.723223 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.723238 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-workload-socket\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:08:24.723223 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.723246 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-token\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:08:24.723531 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.723254 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff59dab7-2302-49dc-a390-661fc2568b3f-istiod-ca-cert\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:08:24.723531 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.723262 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjb5f\" (UniqueName: \"kubernetes.io/projected/ff59dab7-2302-49dc-a390-661fc2568b3f-kube-api-access-bjb5f\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:08:24.723531 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:24.723271 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff59dab7-2302-49dc-a390-661fc2568b3f-istio-data\") on node \"ip-10-0-136-58.ec2.internal\" DevicePath \"\"" Jul 17 02:08:25.298978 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.298942 2564 generic.go:356] "Generic (PLEG): container finished" podID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerID="cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942" exitCode=0 Jul 17 02:08:25.299130 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.299014 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" Jul 17 02:08:25.299130 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.299018 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" event={"ID":"ff59dab7-2302-49dc-a390-661fc2568b3f","Type":"ContainerDied","Data":"cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942"} Jul 17 02:08:25.299130 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.299080 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw" event={"ID":"ff59dab7-2302-49dc-a390-661fc2568b3f","Type":"ContainerDied","Data":"b107a0432e413ed80b6112a47986ea667e6cf555e6545e9954fdc34007faa9fb"} Jul 17 02:08:25.299130 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.299111 2564 scope.go:117] "RemoveContainer" containerID="cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942" Jul 17 02:08:25.309210 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.309191 2564 scope.go:117] "RemoveContainer" containerID="9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5" Jul 17 02:08:25.318017 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.317990 2564 scope.go:117] "RemoveContainer" containerID="cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942" Jul 17 02:08:25.318474 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:08:25.318445 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942\": container with ID starting with cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942 not found: ID does not exist" containerID="cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942" Jul 17 02:08:25.318580 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.318493 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942"} err="failed to get container status \"cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942\": rpc error: code = NotFound desc = could not find container \"cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942\": container with ID starting with cc8407a69dc0f58a5417226a7b0c759132cf955fb8177dc628e4dead93600942 not found: ID does not exist" Jul 17 02:08:25.318580 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.318520 2564 scope.go:117] "RemoveContainer" containerID="9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5" Jul 17 02:08:25.318836 ip-10-0-136-58 kubenswrapper[2564]: E0717 02:08:25.318813 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5\": container with ID starting with 9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5 not found: ID does not exist" containerID="9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5" Jul 17 02:08:25.318889 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.318845 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5"} err="failed to get container status \"9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5\": rpc error: code = NotFound desc = could not find container \"9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5\": container with ID starting with 9a5a5b522dbabaf776c335516c9baaa204c7a5630a289640f3c141b6fa5e3af5 not found: ID does not exist" Jul 17 02:08:25.321202 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.321157 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw"] Jul 17 02:08:25.326433 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:25.326411 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-fd4ae7-openshift-default-84664b775-b79dw"] Jul 17 02:08:26.134779 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:26.134750 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" path="/var/lib/kubelet/pods/ff59dab7-2302-49dc-a390-661fc2568b3f/volumes" Jul 17 02:08:46.074787 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:46.074751 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685db4888-2swfm_0b071c93-8839-46d9-a60e-7a79decb8e10/authorino/0.log" Jul 17 02:08:51.713499 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:51.713465 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f988b7f6f-l87lh_43b71236-8373-4594-b50a-f0dd44ded33c/manager/0.log" Jul 17 02:08:51.821649 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:51.821618 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-65z88_faa1066e-2d83-46f2-aefe-ed11fad61da0/server/0.log" Jul 17 02:08:51.950963 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:51.950923 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5dvqd_7454a501-b0b8-4509-9a4f-a71396becbb3/manager/0.log" Jul 17 02:08:52.325123 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:52.325091 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7_be392a22-c9fe-4f4d-9b92-cfaff6acec2a/manager/0.log" Jul 17 02:08:52.331729 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:52.331709 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7_be392a22-c9fe-4f4d-9b92-cfaff6acec2a/copy-manifests/0.log" Jul 17 02:08:52.697461 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:52.697423 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7954f7d7ff-rspps_140b6f2e-fec4-4fd0-ad32-10bf54a86d6b/maas-api/0.log" Jul 17 02:08:54.040769 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:54.040740 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ls7lb_700b1d31-8695-4938-b909-4a97974e81e5/postgres/0.log" Jul 17 02:08:54.409452 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:54.409427 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 02:08:54.421724 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:54.421704 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6b52l_44020bfd-3d70-4b54-8782-12b022f858bb/ovn-acl-logging/0.log" Jul 17 02:08:54.814236 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:54.814131 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj_02fd5e6c-69f1-47bb-ae16-b848053e2dfa/util/0.log" Jul 17 02:08:54.820730 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:54.820704 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj_02fd5e6c-69f1-47bb-ae16-b848053e2dfa/pull/0.log" Jul 17 02:08:54.827060 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:54.827038 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj_02fd5e6c-69f1-47bb-ae16-b848053e2dfa/extract/0.log" Jul 17 02:08:54.943408 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:54.943380 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9_9b8599f9-59db-4d51-8072-d9ad0265d377/util/0.log" Jul 17 02:08:54.949772 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:54.949738 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9_9b8599f9-59db-4d51-8072-d9ad0265d377/pull/0.log" Jul 17 02:08:54.956095 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:54.956075 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9_9b8599f9-59db-4d51-8072-d9ad0265d377/extract/0.log" Jul 17 02:08:55.066268 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.066152 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj_4715a21c-afee-42d1-9be5-a389f32fb399/extract/0.log" Jul 17 02:08:55.072096 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.072069 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj_4715a21c-afee-42d1-9be5-a389f32fb399/util/0.log" Jul 17 02:08:55.077977 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.077955 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj_4715a21c-afee-42d1-9be5-a389f32fb399/pull/0.log" Jul 17 02:08:55.206582 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.206557 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7_bc530462-42b3-47f6-ba41-ed4ff81c17de/util/0.log" Jul 17 02:08:55.213151 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.213121 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7_bc530462-42b3-47f6-ba41-ed4ff81c17de/pull/0.log" Jul 17 02:08:55.219902 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.219881 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7_bc530462-42b3-47f6-ba41-ed4ff81c17de/extract/0.log" Jul 17 02:08:55.341189 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.341078 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685db4888-2swfm_0b071c93-8839-46d9-a60e-7a79decb8e10/authorino/0.log" Jul 17 02:08:55.463714 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.463686 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4vktj_83788c1d-38da-49f2-bd19-6b48eceb4cc0/manager/0.log" Jul 17 02:08:55.578484 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.578450 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bn4fj_01b979c3-1417-403e-8a15-6fd43b91e033/manager/0.log" Jul 17 02:08:55.692376 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.692343 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-g7cx6_8f55e9a4-88bb-45fe-99c3-7e073d28f7c1/kuadrant-console-plugin/0.log" Jul 17 02:08:55.813430 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.813400 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hsq69_40010bf0-a051-4574-9a4d-3bd215921dce/registry-server/0.log" Jul 17 02:08:55.940205 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:55.940152 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tm2fd_7119e54a-0d4d-4e06-a7bc-1751d08a5f62/manager/0.log" Jul 17 02:08:56.188363 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:56.188330 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bvjxt_a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0/manager/0.log" Jul 17 02:08:56.896003 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:56.895967 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68f596cfdd-tpm2n_92a62262-b71d-419f-be34-a58d94b1c43f/kube-auth-proxy/0.log" Jul 17 02:08:57.020224 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:57.020186 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-dxtsn_d4ac076b-8b5c-4dc1-82db-6fa21f323ecf/istio-proxy/1.log" Jul 17 02:08:57.140242 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:57.140211 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-pr7c6_b568047d-f58b-48d2-b7c8-f5276c47cbd6/payload-pre-processing/0.log" Jul 17 02:08:57.270780 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:57.270753 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-j5v7l_d8be559c-ba5a-4a21-9baf-ea63b397dab6/payload-processing/0.log" Jul 17 02:08:57.729521 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:57.729484 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-ncggs_b7b53d29-4b24-4f16-90a8-3cc35bacfc0c/main/0.log" Jul 17 02:08:57.954464 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:57.954436 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mcms2_397482b6-dd04-4551-be34-0ed3f191ac05/main/0.log" Jul 17 02:08:58.186322 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:58.186293 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-686v9_8f5dfeda-b3ab-4322-83ce-1a7cf2ee10a2/main/0.log" Jul 17 02:08:58.301231 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:08:58.301201 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-gw67n_4fd7c591-b985-404f-a61f-35d178fa81af/main/0.log" Jul 17 02:09:04.033211 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.033157 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-l2j4k/must-gather-bzwf5"] Jul 17 02:09:04.033856 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.033835 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" Jul 17 02:09:04.033948 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.033859 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" Jul 17 02:09:04.033948 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.033881 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" Jul 17 02:09:04.033948 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.033889 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" Jul 17 02:09:04.033948 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.033928 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30968c18-07d5-44bf-9121-bcfc763670fa" containerName="test-curl-7502-139828471363888" Jul 17 02:09:04.033948 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.033939 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="30968c18-07d5-44bf-9121-bcfc763670fa" containerName="test-curl-7502-139828471363888" Jul 17 02:09:04.034237 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.034040 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" Jul 17 02:09:04.034237 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.034056 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff59dab7-2302-49dc-a390-661fc2568b3f" containerName="istio-proxy" Jul 17 02:09:04.034237 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.034066 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="30968c18-07d5-44bf-9121-bcfc763670fa" containerName="test-curl-7502-139828471363888" Jul 17 02:09:04.037437 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.037417 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l2j4k/must-gather-bzwf5" Jul 17 02:09:04.042006 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.041985 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-l2j4k\"/\"default-dockercfg-5srgd\"" Jul 17 02:09:04.042124 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.041988 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l2j4k\"/\"openshift-service-ca.crt\"" Jul 17 02:09:04.042124 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.041993 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l2j4k\"/\"kube-root-ca.crt\"" Jul 17 02:09:04.062135 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.062111 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l2j4k/must-gather-bzwf5"] Jul 17 02:09:04.201099 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.201062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdznp\" (UniqueName: \"kubernetes.io/projected/d3f0502a-3447-4967-bc8c-47c45b8eaee2-kube-api-access-xdznp\") pod \"must-gather-bzwf5\" (UID: \"d3f0502a-3447-4967-bc8c-47c45b8eaee2\") " pod="openshift-must-gather-l2j4k/must-gather-bzwf5" Jul 17 02:09:04.201285 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.201111 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d3f0502a-3447-4967-bc8c-47c45b8eaee2-must-gather-output\") pod \"must-gather-bzwf5\" (UID: \"d3f0502a-3447-4967-bc8c-47c45b8eaee2\") " pod="openshift-must-gather-l2j4k/must-gather-bzwf5" Jul 17 02:09:04.302712 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.302627 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdznp\" (UniqueName: \"kubernetes.io/projected/d3f0502a-3447-4967-bc8c-47c45b8eaee2-kube-api-access-xdznp\") pod \"must-gather-bzwf5\" (UID: \"d3f0502a-3447-4967-bc8c-47c45b8eaee2\") " pod="openshift-must-gather-l2j4k/must-gather-bzwf5" Jul 17 02:09:04.302712 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.302679 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d3f0502a-3447-4967-bc8c-47c45b8eaee2-must-gather-output\") pod \"must-gather-bzwf5\" (UID: \"d3f0502a-3447-4967-bc8c-47c45b8eaee2\") " pod="openshift-must-gather-l2j4k/must-gather-bzwf5" Jul 17 02:09:04.303063 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.303042 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d3f0502a-3447-4967-bc8c-47c45b8eaee2-must-gather-output\") pod \"must-gather-bzwf5\" (UID: \"d3f0502a-3447-4967-bc8c-47c45b8eaee2\") " pod="openshift-must-gather-l2j4k/must-gather-bzwf5" Jul 17 02:09:04.314756 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.314726 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdznp\" (UniqueName: \"kubernetes.io/projected/d3f0502a-3447-4967-bc8c-47c45b8eaee2-kube-api-access-xdznp\") pod \"must-gather-bzwf5\" (UID: \"d3f0502a-3447-4967-bc8c-47c45b8eaee2\") " pod="openshift-must-gather-l2j4k/must-gather-bzwf5" Jul 17 02:09:04.346498 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.346471 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l2j4k/must-gather-bzwf5" Jul 17 02:09:04.498421 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.498392 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l2j4k/must-gather-bzwf5"] Jul 17 02:09:04.499729 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:09:04.499706 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3f0502a_3447_4967_bc8c_47c45b8eaee2.slice/crio-1b973b6246de430e5bc71e5e14f387c375cc87bb6a87265fdcea1a65c1e6c655 WatchSource:0}: Error finding container 1b973b6246de430e5bc71e5e14f387c375cc87bb6a87265fdcea1a65c1e6c655: Status 404 returned error can't find the container with id 1b973b6246de430e5bc71e5e14f387c375cc87bb6a87265fdcea1a65c1e6c655 Jul 17 02:09:04.527322 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:04.527289 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l2j4k/must-gather-bzwf5" event={"ID":"d3f0502a-3447-4967-bc8c-47c45b8eaee2","Type":"ContainerStarted","Data":"1b973b6246de430e5bc71e5e14f387c375cc87bb6a87265fdcea1a65c1e6c655"} Jul 17 02:09:05.549520 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:05.549464 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l2j4k/must-gather-bzwf5" event={"ID":"d3f0502a-3447-4967-bc8c-47c45b8eaee2","Type":"ContainerStarted","Data":"68262576e4c8640997ea46665970ba1db985bfb7fe175929276c04d2195f9ab4"} Jul 17 02:09:06.559194 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:06.559131 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l2j4k/must-gather-bzwf5" event={"ID":"d3f0502a-3447-4967-bc8c-47c45b8eaee2","Type":"ContainerStarted","Data":"27613215a5b8b396d56a3fd4cab9bcfacf93077a3611335cacf7db233efc06d2"} Jul 17 02:09:06.579177 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:06.579122 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-l2j4k/must-gather-bzwf5" podStartSLOduration=1.684418476 podStartE2EDuration="2.57910376s" totalImagesPullingTime="894.685284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 02:09:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:09:06.577122608 +0000 UTC m=+3312.955047016" watchObservedRunningTime="2026-07-17 02:09:06.57910376 +0000 UTC m=+3312.957028167" Jul 17 02:09:07.072345 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:07.072306 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zgf6x_a5f557eb-8315-4fb5-803c-9d0c7961cb86/global-pull-secret-syncer/0.log" Jul 17 02:09:07.224107 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:07.224068 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l9qrz_5602e00b-be85-4eca-adbc-e7852aed270a/konnectivity-agent/0.log" Jul 17 02:09:07.301277 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:07.301243 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-58.ec2.internal_584482ea035b563238d63a3c04b310b9/haproxy/0.log" Jul 17 02:09:11.686457 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.686428 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj_02fd5e6c-69f1-47bb-ae16-b848053e2dfa/extract/0.log" Jul 17 02:09:11.716869 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.716840 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj_02fd5e6c-69f1-47bb-ae16-b848053e2dfa/util/0.log" Jul 17 02:09:11.741976 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.741941 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759cg8wj_02fd5e6c-69f1-47bb-ae16-b848053e2dfa/pull/0.log" Jul 17 02:09:11.770710 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.770657 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9_9b8599f9-59db-4d51-8072-d9ad0265d377/extract/0.log" Jul 17 02:09:11.797264 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.797230 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9_9b8599f9-59db-4d51-8072-d9ad0265d377/util/0.log" Jul 17 02:09:11.820230 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.820111 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0x4tz9_9b8599f9-59db-4d51-8072-d9ad0265d377/pull/0.log" Jul 17 02:09:11.849366 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.849332 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj_4715a21c-afee-42d1-9be5-a389f32fb399/extract/0.log" Jul 17 02:09:11.873212 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.873156 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj_4715a21c-afee-42d1-9be5-a389f32fb399/util/0.log" Jul 17 02:09:11.895257 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.895185 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73m94wj_4715a21c-afee-42d1-9be5-a389f32fb399/pull/0.log" Jul 17 02:09:11.924337 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.924301 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7_bc530462-42b3-47f6-ba41-ed4ff81c17de/extract/0.log" Jul 17 02:09:11.946056 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.945975 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7_bc530462-42b3-47f6-ba41-ed4ff81c17de/util/0.log" Jul 17 02:09:11.973034 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:11.972999 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16zcb7_bc530462-42b3-47f6-ba41-ed4ff81c17de/pull/0.log" Jul 17 02:09:12.475736 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:12.475677 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-685db4888-2swfm_0b071c93-8839-46d9-a60e-7a79decb8e10/authorino/0.log" Jul 17 02:09:12.510432 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:12.510396 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4vktj_83788c1d-38da-49f2-bd19-6b48eceb4cc0/manager/0.log" Jul 17 02:09:12.549278 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:12.549240 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bn4fj_01b979c3-1417-403e-8a15-6fd43b91e033/manager/0.log" Jul 17 02:09:12.573545 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:12.573298 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-g7cx6_8f55e9a4-88bb-45fe-99c3-7e073d28f7c1/kuadrant-console-plugin/0.log" Jul 17 02:09:12.636512 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:12.636446 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hsq69_40010bf0-a051-4574-9a4d-3bd215921dce/registry-server/0.log" Jul 17 02:09:12.831860 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:12.831706 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-tm2fd_7119e54a-0d4d-4e06-a7bc-1751d08a5f62/manager/0.log" Jul 17 02:09:12.941457 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:12.941416 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-bvjxt_a016c5c1-85d6-4dcd-b9c1-d47f0fe319d0/manager/0.log" Jul 17 02:09:14.565366 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:14.565339 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6807f673-2152-4c92-af6f-ad54a23ab100/alertmanager/0.log" Jul 17 02:09:14.595343 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:14.595319 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6807f673-2152-4c92-af6f-ad54a23ab100/config-reloader/0.log" Jul 17 02:09:14.621859 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:14.621832 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6807f673-2152-4c92-af6f-ad54a23ab100/kube-rbac-proxy-web/0.log" Jul 17 02:09:14.641992 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:14.641912 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6807f673-2152-4c92-af6f-ad54a23ab100/kube-rbac-proxy/0.log" Jul 17 02:09:14.673676 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:14.673648 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6807f673-2152-4c92-af6f-ad54a23ab100/kube-rbac-proxy-metric/0.log" Jul 17 02:09:14.697079 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:14.697052 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6807f673-2152-4c92-af6f-ad54a23ab100/prom-label-proxy/0.log" Jul 17 02:09:14.717568 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:14.717532 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6807f673-2152-4c92-af6f-ad54a23ab100/init-config-reloader/0.log" Jul 17 02:09:14.990244 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:14.990193 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p9t9q_31cad27c-0743-490a-bc89-144d3cdec0c4/node-exporter/0.log" Jul 17 02:09:15.009404 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:15.009365 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p9t9q_31cad27c-0743-490a-bc89-144d3cdec0c4/kube-rbac-proxy/0.log" Jul 17 02:09:15.029799 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:15.029770 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p9t9q_31cad27c-0743-490a-bc89-144d3cdec0c4/init-textfile/0.log" Jul 17 02:09:15.398631 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:15.398541 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-cn7dt_9f101251-b498-49bd-b934-fc4fdb6f9110/prometheus-operator/0.log" Jul 17 02:09:15.437738 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:15.437595 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-cn7dt_9f101251-b498-49bd-b934-fc4fdb6f9110/kube-rbac-proxy/0.log" Jul 17 02:09:15.486301 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:15.486260 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69fb9cbc45-8qh7x_7484801f-51e8-4bc2-84b9-2c9d3d2aeb52/telemeter-client/0.log" Jul 17 02:09:15.506762 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:15.506732 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69fb9cbc45-8qh7x_7484801f-51e8-4bc2-84b9-2c9d3d2aeb52/reload/0.log" Jul 17 02:09:15.526179 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:15.526122 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-69fb9cbc45-8qh7x_7484801f-51e8-4bc2-84b9-2c9d3d2aeb52/kube-rbac-proxy/0.log" Jul 17 02:09:17.246778 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.246738 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9"] Jul 17 02:09:17.254868 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.254832 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.262139 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.262109 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9"] Jul 17 02:09:17.274527 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.274486 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-proc\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.274829 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.274808 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-sys\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.275053 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.275036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-lib-modules\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.275236 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.275212 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8wdx\" (UniqueName: \"kubernetes.io/projected/359d1948-3819-44c1-a431-fdfcd13b812f-kube-api-access-m8wdx\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.275363 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.275255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-podres\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.377383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.376274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-sys\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.377383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.376390 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-lib-modules\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.377383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.376428 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8wdx\" (UniqueName: \"kubernetes.io/projected/359d1948-3819-44c1-a431-fdfcd13b812f-kube-api-access-m8wdx\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.377383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.376466 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-podres\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.377383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.376568 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-proc\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.377383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.376706 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-proc\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.377383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.376773 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-sys\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.377383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.376899 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-lib-modules\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.377383 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.377340 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/359d1948-3819-44c1-a431-fdfcd13b812f-podres\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.385606 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.385556 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8wdx\" (UniqueName: \"kubernetes.io/projected/359d1948-3819-44c1-a431-fdfcd13b812f-kube-api-access-m8wdx\") pod \"perf-node-gather-daemonset-tkpc9\" (UID: \"359d1948-3819-44c1-a431-fdfcd13b812f\") " pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.572386 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.572234 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:17.770234 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:17.770198 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9"] Jul 17 02:09:17.771142 ip-10-0-136-58 kubenswrapper[2564]: W0717 02:09:17.771113 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod359d1948_3819_44c1_a431_fdfcd13b812f.slice/crio-5e9473aa57669976bb726a0d7039ed958bd939b786a1c23b21d680118c6b8905 WatchSource:0}: Error finding container 5e9473aa57669976bb726a0d7039ed958bd939b786a1c23b21d680118c6b8905: Status 404 returned error can't find the container with id 5e9473aa57669976bb726a0d7039ed958bd939b786a1c23b21d680118c6b8905 Jul 17 02:09:18.036154 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:18.036098 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78c566b9c6-tl9mf_53d13840-56f9-4214-a14b-05f02ca8a795/console/0.log" Jul 17 02:09:18.078625 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:18.078538 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-7q26l_36908188-7b90-4fa5-b63d-56cf2c3edf48/download-server/0.log" Jul 17 02:09:18.675204 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:18.675146 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" event={"ID":"359d1948-3819-44c1-a431-fdfcd13b812f","Type":"ContainerStarted","Data":"ac72fc645a80678da4e32ac49eb303de93939aa01b50f8b21beb940256a360b5"} Jul 17 02:09:18.675793 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:18.675764 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" event={"ID":"359d1948-3819-44c1-a431-fdfcd13b812f","Type":"ContainerStarted","Data":"5e9473aa57669976bb726a0d7039ed958bd939b786a1c23b21d680118c6b8905"} Jul 17 02:09:18.675929 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:18.675902 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:18.695008 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:18.694953 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" podStartSLOduration=1.694937179 podStartE2EDuration="1.694937179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 02:09:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 02:09:18.690739754 +0000 UTC m=+3325.068664163" watchObservedRunningTime="2026-07-17 02:09:18.694937179 +0000 UTC m=+3325.072861588" Jul 17 02:09:19.696311 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:19.696278 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q7n29_27278acb-c108-4aab-ae3d-24c82e936f6e/dns/0.log" Jul 17 02:09:19.715868 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:19.715841 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q7n29_27278acb-c108-4aab-ae3d-24c82e936f6e/kube-rbac-proxy/0.log" Jul 17 02:09:19.784488 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:19.784460 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7hnr7_991a0470-8bff-4780-8a40-693299c05626/dns-node-resolver/0.log" Jul 17 02:09:20.339831 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:20.339794 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-fd4c74d88-zx8f2_40b01a30-59e8-4950-a1bc-3ce63dc19e9d/registry/0.log" Jul 17 02:09:20.387710 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:20.387678 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jpvtd_86d44f9e-b02e-40ec-863f-b528b36e5f68/node-ca/0.log" Jul 17 02:09:21.615439 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:21.615409 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68f596cfdd-tpm2n_92a62262-b71d-419f-be34-a58d94b1c43f/kube-auth-proxy/0.log" Jul 17 02:09:21.642866 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:21.642831 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-dxtsn_d4ac076b-8b5c-4dc1-82db-6fa21f323ecf/istio-proxy/0.log" Jul 17 02:09:21.708539 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:21.708507 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-dxtsn_d4ac076b-8b5c-4dc1-82db-6fa21f323ecf/istio-proxy/1.log" Jul 17 02:09:21.745740 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:21.745710 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-pr7c6_b568047d-f58b-48d2-b7c8-f5276c47cbd6/payload-pre-processing/0.log" Jul 17 02:09:21.784106 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:21.784078 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-j5v7l_d8be559c-ba5a-4a21-9baf-ea63b397dab6/payload-processing/0.log" Jul 17 02:09:22.518401 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:22.518373 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2b788_8fccaee8-8ef0-4180-90ae-4a6540066918/serve-healthcheck-canary/0.log" Jul 17 02:09:23.074953 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:23.074916 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nx2gc_27bba4b4-b060-45b3-9136-39d03b5e2c00/kube-rbac-proxy/0.log" Jul 17 02:09:23.099365 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:23.099307 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nx2gc_27bba4b4-b060-45b3-9136-39d03b5e2c00/exporter/0.log" Jul 17 02:09:23.123986 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:23.123958 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nx2gc_27bba4b4-b060-45b3-9136-39d03b5e2c00/extractor/0.log" Jul 17 02:09:24.693676 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:24.693644 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-l2j4k/perf-node-gather-daemonset-tkpc9" Jul 17 02:09:25.799183 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:25.797059 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-f988b7f6f-l87lh_43b71236-8373-4594-b50a-f0dd44ded33c/manager/0.log" Jul 17 02:09:25.820993 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:25.820967 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-65z88_faa1066e-2d83-46f2-aefe-ed11fad61da0/server/0.log" Jul 17 02:09:25.945507 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:25.945428 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-5dvqd_7454a501-b0b8-4509-9a4f-a71396becbb3/manager/0.log" Jul 17 02:09:26.173881 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:26.173854 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7_be392a22-c9fe-4f4d-9b92-cfaff6acec2a/manager/0.log" Jul 17 02:09:26.195023 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:26.194997 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56ddbf9bf4-zzwq7_be392a22-c9fe-4f4d-9b92-cfaff6acec2a/copy-manifests/0.log" Jul 17 02:09:28.081342 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:28.081287 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68f596cfdd-tpm2n_92a62262-b71d-419f-be34-a58d94b1c43f/kube-auth-proxy/0.log" Jul 17 02:09:28.100391 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:28.100350 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-dxtsn_d4ac076b-8b5c-4dc1-82db-6fa21f323ecf/istio-proxy/0.log" Jul 17 02:09:28.163746 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:28.163702 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-dxtsn_d4ac076b-8b5c-4dc1-82db-6fa21f323ecf/istio-proxy/1.log" Jul 17 02:09:28.188902 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:28.188868 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-pr7c6_b568047d-f58b-48d2-b7c8-f5276c47cbd6/payload-pre-processing/0.log" Jul 17 02:09:28.224007 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:28.223984 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-j5v7l_d8be559c-ba5a-4a21-9baf-ea63b397dab6/payload-processing/0.log" Jul 17 02:09:29.631688 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:29.631593 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-68f596cfdd-tpm2n_92a62262-b71d-419f-be34-a58d94b1c43f/kube-auth-proxy/0.log" Jul 17 02:09:29.651964 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:29.651932 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-dxtsn_d4ac076b-8b5c-4dc1-82db-6fa21f323ecf/istio-proxy/0.log" Jul 17 02:09:29.713191 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:29.713132 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-dxtsn_d4ac076b-8b5c-4dc1-82db-6fa21f323ecf/istio-proxy/1.log" Jul 17 02:09:29.743994 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:29.743960 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-pr7c6_b568047d-f58b-48d2-b7c8-f5276c47cbd6/payload-pre-processing/0.log" Jul 17 02:09:29.779410 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:29.777300 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-j5v7l_d8be559c-ba5a-4a21-9baf-ea63b397dab6/payload-processing/0.log" Jul 17 02:09:33.888664 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:33.888633 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5f6x_d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb/kube-multus-additional-cni-plugins/0.log" Jul 17 02:09:33.917241 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:33.917203 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5f6x_d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb/egress-router-binary-copy/0.log" Jul 17 02:09:33.938257 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:33.938209 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5f6x_d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb/cni-plugins/0.log" Jul 17 02:09:33.958192 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:33.958138 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5f6x_d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb/bond-cni-plugin/0.log" Jul 17 02:09:33.980042 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:33.980014 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5f6x_d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb/routeoverride-cni/0.log" Jul 17 02:09:34.001084 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:34.001057 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5f6x_d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb/whereabouts-cni-bincopy/0.log" Jul 17 02:09:34.021205 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:34.021179 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-j5f6x_d5a5c7c0-fa10-4f3c-b66e-c329c0b65cbb/whereabouts-cni/0.log" Jul 17 02:09:34.503925 ip-10-0-136-58 kubenswrapper[2564]: I0717 02:09:34.503891 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-z8pf7_5622d70d-5b2e-43f6-9982-b8a931543303/kube-multus/0.log"