Jul 06 23:48:09.638760 ip-10-0-137-188 systemd[1]: Starting Kubernetes Kubelet... Jul 06 23:48:09.975578 ip-10-0-137-188 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:09.975578 ip-10-0-137-188 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 23:48:09.975578 ip-10-0-137-188 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:09.975578 ip-10-0-137-188 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 23:48:09.975578 ip-10-0-137-188 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 23:48:09.976435 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.976359 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 23:48:09.980996 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.980978 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:09.980996 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.980994 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:09.980996 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.980997 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981002 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981005 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981008 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981010 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981013 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981016 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981021 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981025 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981028 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981036 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981040 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981043 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981045 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981048 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981051 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981053 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981056 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981058 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981061 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:09.981102 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981064 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981067 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981069 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981072 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981074 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981077 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981079 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981082 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981085 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981087 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981090 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981092 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981095 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981098 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981101 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981104 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981107 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981110 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981113 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981116 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:09.981596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981119 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981121 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981124 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981126 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981129 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981131 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981134 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981137 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981139 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981143 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981153 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981156 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981159 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981162 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981164 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981167 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981169 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981172 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981176 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981178 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:09.982083 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981181 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981184 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981186 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981189 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981191 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981194 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981197 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981202 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981205 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981208 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981211 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981215 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981217 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981220 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981222 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981225 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981228 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981231 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981233 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:09.982574 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981236 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981238 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981241 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981243 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981246 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981248 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981250 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981253 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981255 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981258 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981261 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981263 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981266 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981269 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981272 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981275 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981278 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:09.983010 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.981281 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:09.983399 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983384 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:09.983399 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983392 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:09.983399 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983396 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:09.983399 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983399 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983402 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983405 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983409 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983420 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983423 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983425 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983428 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983430 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983433 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983435 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983438 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983441 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983444 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983446 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983449 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983451 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983454 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983456 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:09.983512 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983459 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983461 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983464 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983466 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983482 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983485 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983488 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983490 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983493 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983496 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983499 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983501 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983504 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983506 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983509 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983513 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983516 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983519 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983522 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:09.983980 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983525 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983528 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983530 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983533 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983535 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983538 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983541 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983543 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983546 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983548 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983551 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983553 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983555 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983558 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983561 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983563 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983566 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983568 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983571 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983574 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:09.984428 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983576 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983579 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983582 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983585 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983587 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983590 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983592 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983595 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983598 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983600 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983604 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983608 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983611 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983614 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983617 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983620 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983622 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983625 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983628 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:09.984926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983630 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983633 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983635 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983638 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983640 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983643 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983645 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983648 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983650 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983652 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983655 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983659 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983661 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983664 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983667 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983671 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983674 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983677 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.983680 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984054 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:09.985688 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984059 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984063 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984066 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984069 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984072 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984076 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984079 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984081 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984084 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984087 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984090 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984092 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984095 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984098 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984101 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984103 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984107 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984110 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984113 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984117 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:09.986161 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984119 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984122 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984125 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984127 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984130 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984133 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984135 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984138 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984141 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984144 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984146 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984149 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984152 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984154 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984157 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984159 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984162 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984165 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984168 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:09.986680 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984172 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984175 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984178 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984180 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984183 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984186 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984188 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984191 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984193 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984197 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984199 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984202 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984204 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984207 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984209 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984212 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984215 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984217 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984220 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:09.987167 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984223 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984226 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984229 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984232 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984235 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984238 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984242 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984245 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984247 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984250 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984252 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984255 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984257 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984260 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984262 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984265 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984267 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984270 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984272 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:09.987628 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984275 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984277 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984280 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984282 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984284 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984287 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984289 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984292 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984294 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984297 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984299 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984302 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984305 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984307 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984310 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984312 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984315 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984317 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984320 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984323 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.984325 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:09.988093 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984398 2583 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984405 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984411 2583 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984417 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984427 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984432 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984440 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984445 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984448 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984452 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984455 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984459 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984462 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984465 2583 flags.go:64] FLAG: --cgroup-root="" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984468 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984484 2583 flags.go:64] FLAG: --client-ca-file="" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984487 2583 flags.go:64] FLAG: --cloud-provider="external" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984490 2583 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984494 2583 flags.go:64] FLAG: --cluster-domain="" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984497 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984500 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984504 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984511 2583 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984515 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 23:48:09.988631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984518 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984521 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984525 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984528 2583 flags.go:64] FLAG: --contention-profiling="false" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984531 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984534 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984537 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984540 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984544 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984547 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984550 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984553 2583 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984556 2583 flags.go:64] FLAG: --enable-server="true" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984559 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984564 2583 flags.go:64] FLAG: --event-burst="100" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984568 2583 flags.go:64] FLAG: --event-qps="50" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984571 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984574 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984577 2583 flags.go:64] FLAG: --eviction-hard="" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984580 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984583 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984587 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984590 2583 flags.go:64] FLAG: --eviction-soft="" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984593 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984595 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 23:48:09.989214 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984598 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984601 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984604 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984607 2583 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984610 2583 flags.go:64] FLAG: --feature-gates="" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984616 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984619 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984622 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984625 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984628 2583 flags.go:64] FLAG: --healthz-port="10248" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984631 2583 flags.go:64] FLAG: --help="false" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984634 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-137-188.ec2.internal" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984637 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984640 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984643 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984646 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984650 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984652 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984655 2583 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984658 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984661 2583 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984664 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984668 2583 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984671 2583 flags.go:64] FLAG: --kube-reserved="" Jul 06 23:48:09.989852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984674 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984677 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984680 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984687 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984690 2583 flags.go:64] FLAG: --lock-file="" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984693 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984696 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984699 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984704 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984707 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984710 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984712 2583 flags.go:64] FLAG: --logging-format="text" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984715 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984719 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984722 2583 flags.go:64] FLAG: --manifest-url="" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984725 2583 flags.go:64] FLAG: --manifest-url-header="" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984730 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984733 2583 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984737 2583 flags.go:64] FLAG: --max-pods="110" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984740 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984743 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984746 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984749 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984752 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984755 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 23:48:09.990410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984758 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984765 2583 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984768 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984771 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984774 2583 flags.go:64] FLAG: --pod-cidr="" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984777 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984867 2583 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984884 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984890 2583 flags.go:64] FLAG: --pods-per-core="0" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984896 2583 flags.go:64] FLAG: --port="10250" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984903 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984908 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00d6725b286630a53" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984914 2583 flags.go:64] FLAG: --qos-reserved="" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984919 2583 flags.go:64] FLAG: --read-only-port="10255" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984925 2583 flags.go:64] FLAG: --register-node="true" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.984936 2583 flags.go:64] FLAG: --register-with-taints="" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986716 2583 flags.go:64] FLAG: --registry-burst="10" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986726 2583 flags.go:64] FLAG: --registry-qps="5" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986731 2583 flags.go:64] FLAG: --reserved-cpus="" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986735 2583 flags.go:64] FLAG: --reserved-memory="" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986739 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986743 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986746 2583 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986750 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986753 2583 flags.go:64] FLAG: --runonce="false" Jul 06 23:48:09.991019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986756 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986760 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986763 2583 flags.go:64] FLAG: --seccomp-default="false" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986766 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986769 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986773 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986776 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986779 2583 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986782 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986785 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986788 2583 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986790 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986794 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986796 2583 flags.go:64] FLAG: --system-cgroups="" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986799 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986810 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986813 2583 flags.go:64] FLAG: --tls-cert-file="" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986816 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986822 2583 flags.go:64] FLAG: --tls-min-version="" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986824 2583 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986827 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986831 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986834 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986838 2583 flags.go:64] FLAG: --v="2" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986852 2583 flags.go:64] FLAG: --version="false" Jul 06 23:48:09.991652 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986857 2583 flags.go:64] FLAG: --vmodule="" Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986860 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.986864 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.986983 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.986988 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.986991 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.986994 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.986997 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.986999 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987002 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987005 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987008 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987010 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987013 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987016 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987018 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987021 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987024 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987027 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987029 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:09.992282 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987032 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987035 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987038 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987040 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987043 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987046 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987048 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987050 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987053 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987056 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987059 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987061 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987064 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987067 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987069 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987071 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987074 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987079 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987082 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987085 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:09.992779 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987088 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987091 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987094 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987097 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987099 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987102 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987104 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987107 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987110 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987113 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987115 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987118 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987121 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987123 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987125 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987128 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987130 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987134 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987138 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987141 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:09.993275 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987143 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987146 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987150 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987152 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987155 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987157 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987160 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987162 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987165 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987167 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987170 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987173 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987176 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987178 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987181 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987183 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987186 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987188 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987191 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:09.994100 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987194 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987197 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987199 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987202 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987205 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987207 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987209 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987212 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987214 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987217 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987219 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987222 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987225 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987227 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987230 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987233 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987235 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987238 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987240 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:09.995130 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987243 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987247 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987251 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.987254 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.987884 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.995597 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.995619 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.995646 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.995655 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995735 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995744 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995749 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995754 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995758 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:09.995926 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995763 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995768 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995772 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995776 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995781 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995786 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995790 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995795 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995799 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995803 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995808 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995812 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995817 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995821 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995825 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995829 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995834 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995838 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995842 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995846 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:09.996557 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995850 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995857 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995863 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995874 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995880 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995886 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995890 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995895 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995900 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995904 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995908 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995912 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995916 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995921 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995924 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995928 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995933 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995937 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995941 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:09.997284 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995945 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995950 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995954 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995958 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995963 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995967 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995971 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995975 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995980 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995984 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995988 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995992 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.995996 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996001 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996005 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996009 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996013 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996020 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996024 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996028 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:09.997874 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996033 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996037 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996041 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996045 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996049 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996054 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996058 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996062 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996066 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996070 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996074 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996078 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996082 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996086 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996090 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996094 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996099 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996103 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996108 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996112 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:09.998678 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996116 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996120 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996124 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996128 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996132 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996139 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996144 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996149 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996154 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996159 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996164 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996169 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996173 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996178 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:09.999333 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996184 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.996192 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996367 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996374 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996379 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996384 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996389 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996393 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996398 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996403 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996408 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996412 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996417 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996422 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 23:48:09.999827 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996426 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996431 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996435 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996440 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996444 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996448 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996452 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996460 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996466 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996487 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996493 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996497 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996502 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996506 2583 feature_gate.go:328] unrecognized feature gate: Example Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996510 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996515 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996520 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996524 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996528 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 23:48:10.000229 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996532 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996536 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996540 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996545 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996549 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996553 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996557 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996560 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996564 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996568 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996572 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996576 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996580 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996584 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996588 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996592 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996597 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996601 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996605 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996609 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 23:48:10.000727 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996613 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996618 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996622 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996626 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996630 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996635 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996639 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996643 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996647 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996652 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996656 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996660 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996664 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996668 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996672 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996677 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996682 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996686 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996690 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996694 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 23:48:10.001280 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996701 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996706 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996711 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996715 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996719 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996724 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996729 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996733 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996737 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996742 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996746 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996750 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996754 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996758 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996762 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996766 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996772 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996776 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996780 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996784 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 23:48:10.001775 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996789 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996793 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996798 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996802 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996807 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996810 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996814 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:09.996818 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.996826 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.996972 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:09.999774 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.001022 2583 server.go:1019] "Starting client certificate rotation" Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.001126 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:48:10.002316 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.001167 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 23:48:10.003725 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.003708 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:48:10.005174 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.005154 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 23:48:10.014446 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.014428 2583 log.go:25] "Validated CRI v1 runtime API" Jul 06 23:48:10.017516 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.017500 2583 log.go:25] "Validated CRI v1 image API" Jul 06 23:48:10.018425 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.018410 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 23:48:10.020612 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.020594 2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 e62922ab-7f4a-4b1f-8b6c-21b294ccee94:/dev/nvme0n1p4 f8b54dd8-a8d9-41fd-bf59-b28f71203fc4:/dev/nvme0n1p3] Jul 06 23:48:10.020671 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.020611 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 23:48:10.028763 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.028658 2583 manager.go:216] Machine: {Timestamp:2026-07-06 23:48:10.027115741 +0000 UTC m=+0.289787927 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103563 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d03abe767be3edf1af9b9a6bc4b83 SystemUUID:ec2d03ab-e767-be3e-df1a-f9b9a6bc4b83 BootID:16c3649a-cf7b-43ec-bf05-da02ebdc8bff Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:61:d7:66:f3:e3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:61:d7:66:f3:e3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:78:d4:7f:62:65 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 23:48:10.028763 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.028753 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 23:48:10.028876 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.028830 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 23:48:10.031309 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.031282 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 23:48:10.031441 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.031311 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-188.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 23:48:10.031814 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.031803 2583 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 23:48:10.031850 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.031814 2583 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 23:48:10.031850 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.031821 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:48:10.031919 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.031903 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:48:10.032458 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.032447 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 23:48:10.033773 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.033762 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 23:48:10.036641 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.036630 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:48:10.036754 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.036745 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 23:48:10.036784 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.036776 2583 kubelet.go:486] "Attempting to sync node with API server" Jul 06 23:48:10.036811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.036786 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 23:48:10.036811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.036800 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 23:48:10.036811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.036809 2583 kubelet.go:398] "Adding apiserver pod source" Jul 06 23:48:10.036894 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.036817 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 23:48:10.037129 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.037118 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:48:10.037173 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.037137 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 23:48:10.039089 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.039075 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 23:48:10.041034 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.041018 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 23:48:10.041085 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.041051 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 23:48:10.042310 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042295 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 23:48:10.042368 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042317 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 23:48:10.042368 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042324 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 23:48:10.042368 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042330 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 23:48:10.042368 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042339 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 23:48:10.042368 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042345 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 23:48:10.042368 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042353 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 23:48:10.042368 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042358 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 23:48:10.042674 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042377 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 23:48:10.042674 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042392 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 23:48:10.042674 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042408 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 23:48:10.042674 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.042413 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 23:48:10.043039 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.043029 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 23:48:10.043088 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.043041 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 23:48:10.046259 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.046244 2583 server.go:1268] "Started kubelet" Jul 06 23:48:10.046369 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.046326 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 23:48:10.046560 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.046518 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 23:48:10.046675 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.046649 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 23:48:10.047127 ip-10-0-137-188 systemd[1]: Started Kubernetes Kubelet. Jul 06 23:48:10.047384 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.047355 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 23:48:10.047694 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.047596 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-188.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 23:48:10.047755 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.047697 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 23:48:10.048616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.048602 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 23:48:10.048854 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.048832 2583 server.go:310] "Adding debug handlers to kubelet server" Jul 06 23:48:10.051946 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.051925 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 23:48:10.055184 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.055169 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 23:48:10.055259 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.055183 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 23:48:10.055315 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.055301 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.055744 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.054791 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-188.ec2.internal.18bfd832c6240752 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-188.ec2.internal,UID:ip-10-0-137-188.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-188.ec2.internal,},FirstTimestamp:2026-07-06 23:48:10.046220114 +0000 UTC m=+0.308892298,LastTimestamp:2026-07-06 23:48:10.046220114 +0000 UTC m=+0.308892298,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-188.ec2.internal,}" Jul 06 23:48:10.055744 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.055738 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 23:48:10.055861 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.055754 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 23:48:10.055861 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.055738 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 23:48:10.055955 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.055862 2583 reconstruct.go:97] "Volume reconstruction finished" Jul 06 23:48:10.055955 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.055870 2583 reconciler.go:29] "Reconciler: start to sync state" Jul 06 23:48:10.056176 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.056161 2583 factory.go:153] Registering CRI-O factory Jul 06 23:48:10.056242 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.056204 2583 factory.go:223] Registration of the crio container factory successfully Jul 06 23:48:10.056287 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.056258 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 23:48:10.056287 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.056268 2583 factory.go:55] Registering systemd factory Jul 06 23:48:10.056287 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.056276 2583 factory.go:223] Registration of the systemd container factory successfully Jul 06 23:48:10.056418 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.056297 2583 factory.go:103] Registering Raw factory Jul 06 23:48:10.056418 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.056311 2583 manager.go:1196] Started watching for new ooms in manager Jul 06 23:48:10.057019 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.057002 2583 manager.go:349] Starting recovery of all containers Jul 06 23:48:10.058209 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.058183 2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-188.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 23:48:10.059361 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.059336 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bjxwr" Jul 06 23:48:10.067609 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.067584 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 23:48:10.067782 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.067762 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bjxwr" Jul 06 23:48:10.070175 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.070049 2583 manager.go:354] Recovery completed Jul 06 23:48:10.071815 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.071791 2583 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 06 23:48:10.074674 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.074662 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:10.076641 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.076626 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:10.076713 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.076658 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:10.076713 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.076674 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:10.077217 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.077204 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 23:48:10.077217 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.077217 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 23:48:10.077330 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.077235 2583 state_mem.go:36] "Initialized new in-memory state store" Jul 06 23:48:10.077330 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.077206 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:10.078234 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.078222 2583 policy_none.go:49] "None policy: Start" Jul 06 23:48:10.078309 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.078239 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 23:48:10.078309 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.078253 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 23:48:10.078309 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.078286 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:10.078442 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.078313 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:10.078442 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.078329 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:10.079410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.079398 2583 policy_none.go:47] "Start" Jul 06 23:48:10.118993 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.118979 2583 manager.go:337] "Starting Device Plugin manager" Jul 06 23:48:10.119095 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.119010 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 23:48:10.119095 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.119019 2583 server.go:87] "Starting device plugin registration server" Jul 06 23:48:10.119238 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.119228 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 23:48:10.119291 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.119240 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 23:48:10.119346 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.119328 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 23:48:10.119418 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.119402 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 23:48:10.119418 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.119413 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 23:48:10.120131 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.120111 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 23:48:10.120190 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.120166 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.173362 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.173326 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 23:48:10.174536 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.174520 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 23:48:10.174618 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.174562 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 23:48:10.174618 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.174591 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 23:48:10.174701 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.174632 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 23:48:10.177101 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.177082 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:10.220084 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.220054 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:10.222237 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.222216 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:10.222360 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.222250 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:10.222360 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.222260 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:10.222360 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.222283 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.234159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.234109 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.234159 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.234131 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-188.ec2.internal\": node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.250943 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.250905 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.275708 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.275682 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal"] Jul 06 23:48:10.275795 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.275747 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:10.277181 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.277162 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:10.277270 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.277192 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:10.277270 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.277203 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:10.278372 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.278360 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:10.278525 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.278513 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.278560 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.278538 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:10.279063 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.279048 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:10.279156 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.279073 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:10.279156 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.279083 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:10.279156 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.279049 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:10.279156 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.279155 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:10.279275 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.279168 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:10.280150 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.280138 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.280191 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.280167 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 23:48:10.280818 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.280803 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientMemory" Jul 06 23:48:10.280900 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.280832 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 23:48:10.281291 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.281277 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeHasSufficientPID" Jul 06 23:48:10.308141 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.308120 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-188.ec2.internal\" not found" node="ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.312602 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.312586 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-188.ec2.internal\" not found" node="ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.351649 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.351620 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.358097 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.358068 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/769b25d8e28cec488cb47ae7f8f5c1fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal\" (UID: \"769b25d8e28cec488cb47ae7f8f5c1fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.358171 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.358104 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/769b25d8e28cec488cb47ae7f8f5c1fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal\" (UID: \"769b25d8e28cec488cb47ae7f8f5c1fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.358171 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.358121 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ad2d9a7286370db5ed9312271a4a122-config\") pod \"kube-apiserver-proxy-ip-10-0-137-188.ec2.internal\" (UID: \"8ad2d9a7286370db5ed9312271a4a122\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.452539 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.452512 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.459378 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.459357 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/769b25d8e28cec488cb47ae7f8f5c1fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal\" (UID: \"769b25d8e28cec488cb47ae7f8f5c1fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.459475 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.459390 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/769b25d8e28cec488cb47ae7f8f5c1fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal\" (UID: \"769b25d8e28cec488cb47ae7f8f5c1fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.459475 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.459431 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ad2d9a7286370db5ed9312271a4a122-config\") pod \"kube-apiserver-proxy-ip-10-0-137-188.ec2.internal\" (UID: \"8ad2d9a7286370db5ed9312271a4a122\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.459609 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.459472 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8ad2d9a7286370db5ed9312271a4a122-config\") pod \"kube-apiserver-proxy-ip-10-0-137-188.ec2.internal\" (UID: \"8ad2d9a7286370db5ed9312271a4a122\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.459609 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.459470 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/769b25d8e28cec488cb47ae7f8f5c1fc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal\" (UID: \"769b25d8e28cec488cb47ae7f8f5c1fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.459609 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.459471 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/769b25d8e28cec488cb47ae7f8f5c1fc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal\" (UID: \"769b25d8e28cec488cb47ae7f8f5c1fc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.553775 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.553691 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.614806 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.614767 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.617684 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:10.617416 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal" Jul 06 23:48:10.654469 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.654440 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.758391 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.758345 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.859584 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.859508 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:10.960491 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:10.960435 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:11.000863 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.000817 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 23:48:11.001434 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.001003 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 23:48:11.061138 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:11.061105 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:11.071826 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.071782 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 23:43:10 +0000 UTC" deadline="2028-04-21 07:39:29.07200384 +0000 UTC" Jul 06 23:48:11.071826 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.071826 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15703h51m18.000183481s" Jul 06 23:48:11.141198 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:11.141160 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769b25d8e28cec488cb47ae7f8f5c1fc.slice/crio-7594099a4f0189431fef0ccf3d7700b4511fd87d2efde90a7c3d4429de4215d7 WatchSource:0}: Error finding container 7594099a4f0189431fef0ccf3d7700b4511fd87d2efde90a7c3d4429de4215d7: Status 404 returned error can't find the container with id 7594099a4f0189431fef0ccf3d7700b4511fd87d2efde90a7c3d4429de4215d7 Jul 06 23:48:11.145562 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.145534 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:48:11.162034 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:11.162002 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:11.178276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.178228 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" event={"ID":"769b25d8e28cec488cb47ae7f8f5c1fc","Type":"ContainerStarted","Data":"7594099a4f0189431fef0ccf3d7700b4511fd87d2efde90a7c3d4429de4215d7"} Jul 06 23:48:11.238186 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:11.238157 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad2d9a7286370db5ed9312271a4a122.slice/crio-1b147c508a0dbd5374d4207f57c0c83829067bed3282fdbbfdc4f6c12091f553 WatchSource:0}: Error finding container 1b147c508a0dbd5374d4207f57c0c83829067bed3282fdbbfdc4f6c12091f553: Status 404 returned error can't find the container with id 1b147c508a0dbd5374d4207f57c0c83829067bed3282fdbbfdc4f6c12091f553 Jul 06 23:48:11.263157 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:11.263128 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:11.363294 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:11.363261 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-188.ec2.internal\" not found" Jul 06 23:48:11.419081 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.419006 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:11.456880 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.456852 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal" Jul 06 23:48:11.467070 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.467054 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:48:11.468699 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.468686 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" Jul 06 23:48:11.478745 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.478731 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 23:48:11.537176 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.537152 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:11.620922 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:11.620900 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:12.038278 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.038246 2583 apiserver.go:52] "Watching apiserver" Jul 06 23:48:12.043432 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.043405 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 23:48:12.045620 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.045589 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-sfsws","kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal","openshift-cluster-node-tuning-operator/tuned-nkgkq","openshift-image-registry/node-ca-j9bq7","openshift-multus/multus-additional-cni-plugins-zkdd7","openshift-multus/network-metrics-daemon-nhbt9","openshift-network-operator/iptables-alerter-rl9tb","openshift-ovn-kubernetes/ovnkube-node-j7xl5","kube-system/global-pull-secret-syncer-vcbhn","kube-system/konnectivity-agent-8t9lx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q","openshift-dns/node-resolver-sm2dg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal","openshift-multus/multus-ssgq9"] Jul 06 23:48:12.047204 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.047180 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.048483 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.048385 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.050683 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.049876 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:48:12.050683 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.050238 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6hpdl\"" Jul 06 23:48:12.052406 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.051488 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 23:48:12.052406 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.051534 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 23:48:12.052406 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.051711 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 23:48:12.052406 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.051803 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 23:48:12.052406 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.051977 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 23:48:12.052406 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.052019 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 23:48:12.052406 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.052182 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5lzfl\"" Jul 06 23:48:12.055714 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.054623 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.056638 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.056619 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 23:48:12.056742 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.056699 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:12.056828 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.056799 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:12.057131 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.057110 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.058721 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.058698 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:48:12.059171 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.058882 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 23:48:12.059171 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.058942 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 23:48:12.059299 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.059244 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bnj6b\"" Jul 06 23:48:12.059504 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.059481 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p2wq7\"" Jul 06 23:48:12.059767 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.059746 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:12.060048 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.060024 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:12.061981 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.060192 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 23:48:12.061981 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.060230 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 23:48:12.062495 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.062470 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.064792 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.064765 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 23:48:12.065091 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.065070 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 23:48:12.065299 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.065278 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z4bjs\"" Jul 06 23:48:12.065537 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.065515 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 23:48:12.066149 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.066132 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.067771 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.067745 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-kubernetes\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.067868 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.067792 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-system-cni-dir\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.067868 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.067822 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-os-release\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.067868 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.067847 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-var-lib-cni-multus\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.068032 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.067875 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv29j\" (UniqueName: \"kubernetes.io/projected/f0834213-feba-4bf3-b79d-cffe935e7be6-kube-api-access-lv29j\") pod \"iptables-alerter-rl9tb\" (UID: \"f0834213-feba-4bf3-b79d-cffe935e7be6\") " pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068376 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-multus-socket-dir-parent\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068418 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-etc-kubernetes\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068444 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-host\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068470 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-hostroot\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068496 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-run-multus-certs\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068520 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-sysctl-d\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068546 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-run\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068601 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-var-lib-kubelet\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068625 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44d0ce9e-14d3-4d97-a170-dbb4b2772235-tmp\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068673 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-socket-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.068733 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068696 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-modprobe-d\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068753 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-sys-fs\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068791 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0834213-feba-4bf3-b79d-cffe935e7be6-host-slash\") pod \"iptables-alerter-rl9tb\" (UID: \"f0834213-feba-4bf3-b79d-cffe935e7be6\") " pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068831 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-cnibin\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068863 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-run-netns\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068906 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-sys\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068941 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068962 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.068973 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-registration-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069000 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0834213-feba-4bf3-b79d-cffe935e7be6-iptables-alerter-script\") pod \"iptables-alerter-rl9tb\" (UID: \"f0834213-feba-4bf3-b79d-cffe935e7be6\") " pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069030 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4742585e-894a-425f-b770-15e50d8a2d82-cni-binary-copy\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069055 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-run-k8s-cni-cncf-io\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069080 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-sysctl-conf\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069105 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-systemd\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069153 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-lib-modules\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069181 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-tuned\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069199 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069205 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-multus-cni-dir\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.069379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069230 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-var-lib-cni-bin\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069273 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069260 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-var-lib-kubelet\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069310 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4742585e-894a-425f-b770-15e50d8a2d82-multus-daemon-config\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069336 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2bms\" (UniqueName: \"kubernetes.io/projected/44d0ce9e-14d3-4d97-a170-dbb4b2772235-kube-api-access-b2bms\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069420 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-device-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-etc-selinux\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069463 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-multus-conf-dir\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069486 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069573 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hqs45\"" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069459 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069721 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qg9t\" (UniqueName: \"kubernetes.io/projected/4742585e-894a-425f-b770-15e50d8a2d82-kube-api-access-6qg9t\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069748 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-sysconfig\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.070276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.069782 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nthnd\" (UniqueName: \"kubernetes.io/projected/cfb52deb-1807-4c14-ab9c-a19db334eff1-kube-api-access-nthnd\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.070978 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.070815 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 23:48:12.074625 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.074471 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.074625 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.074560 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:12.075924 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.075905 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:12.077259 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.077240 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.078165 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.078122 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 23:48:12.078165 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.078157 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 23:48:12.078434 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.078414 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qcmn4\"" Jul 06 23:48:12.079417 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.079398 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8nlrp\"" Jul 06 23:48:12.079746 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.079698 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 23:48:12.080142 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.080122 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 23:48:12.081424 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.081401 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.083917 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.083878 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 23:48:12.084139 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.084121 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wrd4j\"" Jul 06 23:48:12.084403 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.084385 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 23:48:12.092783 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.092764 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8c6q" Jul 06 23:48:12.100544 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.100522 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w8c6q" Jul 06 23:48:12.157167 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.157144 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 23:48:12.171027 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.170976 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-kubelet\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.171027 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171019 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-run-systemd\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.171220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171044 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-cni-bin\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.171220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171075 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv29j\" (UniqueName: \"kubernetes.io/projected/f0834213-feba-4bf3-b79d-cffe935e7be6-kube-api-access-lv29j\") pod \"iptables-alerter-rl9tb\" (UID: \"f0834213-feba-4bf3-b79d-cffe935e7be6\") " pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.171220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171104 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-etc-kubernetes\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.171220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171156 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-host\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.171220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171169 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-etc-kubernetes\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.171220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171196 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/673c0dc9-eae9-4d18-ac43-825856e88954-hosts-file\") pod \"node-resolver-sm2dg\" (UID: \"673c0dc9-eae9-4d18-ac43-825856e88954\") " pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.171489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171225 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-run-ovn-kubernetes\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.171489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171256 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-run-multus-certs\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.171489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171293 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-run\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.171489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171320 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/145be0a6-160e-481e-987c-fb1af73e4257-agent-certs\") pod \"konnectivity-agent-8t9lx\" (UID: \"145be0a6-160e-481e-987c-fb1af73e4257\") " pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:12.171489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171345 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/145be0a6-160e-481e-987c-fb1af73e4257-konnectivity-ca\") pod \"konnectivity-agent-8t9lx\" (UID: \"145be0a6-160e-481e-987c-fb1af73e4257\") " pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:12.171489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171371 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-os-release\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.171489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171395 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-systemd-units\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.171489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171452 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-run-multus-certs\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.171489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171475 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-cni-netd\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171527 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-host\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171516 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-modprobe-d\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171564 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-run\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171639 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-system-cni-dir\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171680 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4rhw\" (UniqueName: \"kubernetes.io/projected/d751b2b6-b78f-41b3-b895-5d559e9926d1-kube-api-access-x4rhw\") pod \"node-ca-j9bq7\" (UID: \"d751b2b6-b78f-41b3-b895-5d559e9926d1\") " pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171684 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-modprobe-d\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171714 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0834213-feba-4bf3-b79d-cffe935e7be6-host-slash\") pod \"iptables-alerter-rl9tb\" (UID: \"f0834213-feba-4bf3-b79d-cffe935e7be6\") " pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171753 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-run-netns\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171776 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-sys\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171799 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-registration-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171812 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f0834213-feba-4bf3-b79d-cffe935e7be6-host-slash\") pod \"iptables-alerter-rl9tb\" (UID: \"f0834213-feba-4bf3-b79d-cffe935e7be6\") " pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171834 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d751b2b6-b78f-41b3-b895-5d559e9926d1-host\") pod \"node-ca-j9bq7\" (UID: \"d751b2b6-b78f-41b3-b895-5d559e9926d1\") " pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171853 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-registration-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171860 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-sys\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171888 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-var-lib-openvswitch\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171915 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-run-k8s-cni-cncf-io\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.171998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171949 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-sysctl-conf\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171954 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-run-k8s-cni-cncf-io\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171970 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-lib-modules\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171903 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-run-netns\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.171995 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-tuned\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172020 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-multus-cni-dir\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172043 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-var-lib-cni-bin\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172064 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-device-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172085 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-etc-selinux\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172094 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-sysctl-conf\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172113 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172113 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-lib-modules\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172142 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-var-lib-cni-bin\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172172 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-device-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172213 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-multus-conf-dir\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172238 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-etc-selinux\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172239 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9d9114c9-cf23-440f-a966-794fe592ced5-kubelet-config\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.172789 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172255 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-multus-cni-dir\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172275 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-multus-conf-dir\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172282 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxh5g\" (UniqueName: \"kubernetes.io/projected/5e0b8500-f7ec-40ad-885d-e358f1195532-kube-api-access-nxh5g\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172309 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172319 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-system-cni-dir\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-os-release\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-multus-socket-dir-parent\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172385 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-system-cni-dir\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172397 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60463747-bcd4-4066-9b4b-09ce3cf37a1f-env-overrides\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-hostroot\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172443 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-os-release\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172447 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-sysctl-d\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172477 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-var-lib-kubelet\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172544 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-var-lib-kubelet\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172550 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-multus-socket-dir-parent\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172563 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-sysctl-d\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172579 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44d0ce9e-14d3-4d97-a170-dbb4b2772235-tmp\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172589 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-hostroot\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.173534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172625 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-socket-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172647 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172663 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq8jx\" (UniqueName: \"kubernetes.io/projected/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-kube-api-access-mq8jx\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-sys-fs\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172707 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9d9114c9-cf23-440f-a966-794fe592ced5-dbus\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172721 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d751b2b6-b78f-41b3-b895-5d559e9926d1-serviceca\") pod \"node-ca-j9bq7\" (UID: \"d751b2b6-b78f-41b3-b895-5d559e9926d1\") " pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172735 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-etc-openvswitch\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-node-log\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172765 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172782 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-cnibin\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172798 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172817 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/673c0dc9-eae9-4d18-ac43-825856e88954-tmp-dir\") pod \"node-resolver-sm2dg\" (UID: \"673c0dc9-eae9-4d18-ac43-825856e88954\") " pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172833 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-log-socket\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172851 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg88l\" (UniqueName: \"kubernetes.io/projected/60463747-bcd4-4066-9b4b-09ce3cf37a1f-kube-api-access-vg88l\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172888 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0834213-feba-4bf3-b79d-cffe935e7be6-iptables-alerter-script\") pod \"iptables-alerter-rl9tb\" (UID: \"f0834213-feba-4bf3-b79d-cffe935e7be6\") " pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172920 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-socket-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.174159 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172942 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-cnibin\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172977 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4742585e-894a-425f-b770-15e50d8a2d82-cni-binary-copy\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172988 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-sys-fs\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.172944 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cfb52deb-1807-4c14-ab9c-a19db334eff1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-systemd\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173033 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-cnibin\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173073 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-systemd\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173109 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173134 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-run-openvswitch\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173168 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60463747-bcd4-4066-9b4b-09ce3cf37a1f-ovnkube-config\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173197 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60463747-bcd4-4066-9b4b-09ce3cf37a1f-ovnkube-script-lib\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-var-lib-kubelet\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173249 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4742585e-894a-425f-b770-15e50d8a2d82-multus-daemon-config\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2bms\" (UniqueName: \"kubernetes.io/projected/44d0ce9e-14d3-4d97-a170-dbb4b2772235-kube-api-access-b2bms\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173317 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-slash\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173342 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-run-netns\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173368 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-run-ovn\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.174942 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173395 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qg9t\" (UniqueName: \"kubernetes.io/projected/4742585e-894a-425f-b770-15e50d8a2d82-kube-api-access-6qg9t\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173419 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-sysconfig\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173444 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nthnd\" (UniqueName: \"kubernetes.io/projected/cfb52deb-1807-4c14-ab9c-a19db334eff1-kube-api-access-nthnd\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173471 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ssw9\" (UniqueName: \"kubernetes.io/projected/673c0dc9-eae9-4d18-ac43-825856e88954-kube-api-access-5ssw9\") pod \"node-resolver-sm2dg\" (UID: \"673c0dc9-eae9-4d18-ac43-825856e88954\") " pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173570 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4742585e-894a-425f-b770-15e50d8a2d82-cni-binary-copy\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173580 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60463747-bcd4-4066-9b4b-09ce3cf37a1f-ovn-node-metrics-cert\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173612 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-kubernetes\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173639 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-var-lib-kubelet\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-var-lib-cni-multus\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173670 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f0834213-feba-4bf3-b79d-cffe935e7be6-iptables-alerter-script\") pod \"iptables-alerter-rl9tb\" (UID: \"f0834213-feba-4bf3-b79d-cffe935e7be6\") " pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173671 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173718 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173745 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.173839 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-sysconfig\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.174043 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4742585e-894a-425f-b770-15e50d8a2d82-multus-daemon-config\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.174054 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-kubernetes\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.175811 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.174107 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4742585e-894a-425f-b770-15e50d8a2d82-host-var-lib-cni-multus\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.176670 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.176427 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44d0ce9e-14d3-4d97-a170-dbb4b2772235-tmp\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.176670 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.176620 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/44d0ce9e-14d3-4d97-a170-dbb4b2772235-etc-tuned\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.182175 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.182152 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:12.182292 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.182182 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:12.182292 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.182196 2583 projected.go:196] Error preparing data for projected volume kube-api-access-hxbgj for pod openshift-network-diagnostics/network-check-target-sfsws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:12.182292 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.182265 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj podName:5b6424b2-15c0-472d-8e25-4e69aab7538a nodeName:}" failed. No retries permitted until 2026-07-06 23:48:12.682242849 +0000 UTC m=+2.944915031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hxbgj" (UniqueName: "kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj") pod "network-check-target-sfsws" (UID: "5b6424b2-15c0-472d-8e25-4e69aab7538a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:12.183491 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.183442 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal" event={"ID":"8ad2d9a7286370db5ed9312271a4a122","Type":"ContainerStarted","Data":"1b147c508a0dbd5374d4207f57c0c83829067bed3282fdbbfdc4f6c12091f553"} Jul 06 23:48:12.183698 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.183673 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv29j\" (UniqueName: \"kubernetes.io/projected/f0834213-feba-4bf3-b79d-cffe935e7be6-kube-api-access-lv29j\") pod \"iptables-alerter-rl9tb\" (UID: \"f0834213-feba-4bf3-b79d-cffe935e7be6\") " pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.186262 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.186165 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qg9t\" (UniqueName: \"kubernetes.io/projected/4742585e-894a-425f-b770-15e50d8a2d82-kube-api-access-6qg9t\") pod \"multus-ssgq9\" (UID: \"4742585e-894a-425f-b770-15e50d8a2d82\") " pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.186262 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.186233 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2bms\" (UniqueName: \"kubernetes.io/projected/44d0ce9e-14d3-4d97-a170-dbb4b2772235-kube-api-access-b2bms\") pod \"tuned-nkgkq\" (UID: \"44d0ce9e-14d3-4d97-a170-dbb4b2772235\") " pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.186570 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.186541 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nthnd\" (UniqueName: \"kubernetes.io/projected/cfb52deb-1807-4c14-ab9c-a19db334eff1-kube-api-access-nthnd\") pod \"aws-ebs-csi-driver-node-h566q\" (UID: \"cfb52deb-1807-4c14-ab9c-a19db334eff1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.274503 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274443 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxh5g\" (UniqueName: \"kubernetes.io/projected/5e0b8500-f7ec-40ad-885d-e358f1195532-kube-api-access-nxh5g\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274520 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60463747-bcd4-4066-9b4b-09ce3cf37a1f-env-overrides\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274553 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274574 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq8jx\" (UniqueName: \"kubernetes.io/projected/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-kube-api-access-mq8jx\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274596 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9d9114c9-cf23-440f-a966-794fe592ced5-dbus\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274621 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d751b2b6-b78f-41b3-b895-5d559e9926d1-serviceca\") pod \"node-ca-j9bq7\" (UID: \"d751b2b6-b78f-41b3-b895-5d559e9926d1\") " pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274640 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-etc-openvswitch\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274655 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-node-log\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274673 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274692 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/673c0dc9-eae9-4d18-ac43-825856e88954-tmp-dir\") pod \"node-resolver-sm2dg\" (UID: \"673c0dc9-eae9-4d18-ac43-825856e88954\") " pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.274703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274707 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-log-socket\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274722 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg88l\" (UniqueName: \"kubernetes.io/projected/60463747-bcd4-4066-9b4b-09ce3cf37a1f-kube-api-access-vg88l\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274739 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-cnibin\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274760 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274782 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-run-openvswitch\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274829 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60463747-bcd4-4066-9b4b-09ce3cf37a1f-ovnkube-config\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274846 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60463747-bcd4-4066-9b4b-09ce3cf37a1f-ovnkube-script-lib\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274865 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-slash\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274886 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-run-netns\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-run-ovn\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274935 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ssw9\" (UniqueName: \"kubernetes.io/projected/673c0dc9-eae9-4d18-ac43-825856e88954-kube-api-access-5ssw9\") pod \"node-resolver-sm2dg\" (UID: \"673c0dc9-eae9-4d18-ac43-825856e88954\") " pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274961 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60463747-bcd4-4066-9b4b-09ce3cf37a1f-ovn-node-metrics-cert\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.274997 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275042 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275061 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-kubelet\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275092 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-run-systemd\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.275230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275121 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-cni-bin\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275153 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/673c0dc9-eae9-4d18-ac43-825856e88954-hosts-file\") pod \"node-resolver-sm2dg\" (UID: \"673c0dc9-eae9-4d18-ac43-825856e88954\") " pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275179 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-run-ovn-kubernetes\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275205 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/145be0a6-160e-481e-987c-fb1af73e4257-agent-certs\") pod \"konnectivity-agent-8t9lx\" (UID: \"145be0a6-160e-481e-987c-fb1af73e4257\") " pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275229 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/145be0a6-160e-481e-987c-fb1af73e4257-konnectivity-ca\") pod \"konnectivity-agent-8t9lx\" (UID: \"145be0a6-160e-481e-987c-fb1af73e4257\") " pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275262 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-os-release\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275290 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-systemd-units\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275314 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-cni-netd\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275343 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-system-cni-dir\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275341 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-etc-openvswitch\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275371 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4rhw\" (UniqueName: \"kubernetes.io/projected/d751b2b6-b78f-41b3-b895-5d559e9926d1-kube-api-access-x4rhw\") pod \"node-ca-j9bq7\" (UID: \"d751b2b6-b78f-41b3-b895-5d559e9926d1\") " pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275401 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d751b2b6-b78f-41b3-b895-5d559e9926d1-host\") pod \"node-ca-j9bq7\" (UID: \"d751b2b6-b78f-41b3-b895-5d559e9926d1\") " pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275405 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-node-log\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275428 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-var-lib-openvswitch\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275448 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275467 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9d9114c9-cf23-440f-a966-794fe592ced5-kubelet-config\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.276212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275625 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9d9114c9-cf23-440f-a966-794fe592ced5-kubelet-config\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275676 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-cnibin\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275687 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d751b2b6-b78f-41b3-b895-5d559e9926d1-serviceca\") pod \"node-ca-j9bq7\" (UID: \"d751b2b6-b78f-41b3-b895-5d559e9926d1\") " pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275738 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-log-socket\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.275820 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/673c0dc9-eae9-4d18-ac43-825856e88954-tmp-dir\") pod \"node-resolver-sm2dg\" (UID: \"673c0dc9-eae9-4d18-ac43-825856e88954\") " pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276117 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9d9114c9-cf23-440f-a966-794fe592ced5-dbus\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276117 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-run-systemd\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276152 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/60463747-bcd4-4066-9b4b-09ce3cf37a1f-env-overrides\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276245 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276334 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-cni-bin\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276390 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/673c0dc9-eae9-4d18-ac43-825856e88954-hosts-file\") pod \"node-resolver-sm2dg\" (UID: \"673c0dc9-eae9-4d18-ac43-825856e88954\") " pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276403 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-cni-binary-copy\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276437 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-run-ovn-kubernetes\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276491 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-system-cni-dir\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276640 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/60463747-bcd4-4066-9b4b-09ce3cf37a1f-ovnkube-config\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276702 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-run-openvswitch\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276711 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-var-lib-openvswitch\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276749 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d751b2b6-b78f-41b3-b895-5d559e9926d1-host\") pod \"node-ca-j9bq7\" (UID: \"d751b2b6-b78f-41b3-b895-5d559e9926d1\") " pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.277122 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.276829 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.276877 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.276893 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret podName:9d9114c9-cf23-440f-a966-794fe592ced5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:12.776873695 +0000 UTC m=+3.039545884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret") pod "global-pull-secret-syncer-vcbhn" (UID: "9d9114c9-cf23-440f-a966-794fe592ced5") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.276958 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.276989 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs podName:5e0b8500-f7ec-40ad-885d-e358f1195532 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:12.776978514 +0000 UTC m=+3.039650692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs") pod "network-metrics-daemon-nhbt9" (UID: "5e0b8500-f7ec-40ad-885d-e358f1195532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.277029 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-kubelet\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.277251 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-os-release\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.277338 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-run-ovn\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.277369 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-run-netns\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.277408 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-slash\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.277459 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-systemd-units\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.277500 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/60463747-bcd4-4066-9b4b-09ce3cf37a1f-host-cni-netd\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.277639 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.278075 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.277813 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/145be0a6-160e-481e-987c-fb1af73e4257-konnectivity-ca\") pod \"konnectivity-agent-8t9lx\" (UID: \"145be0a6-160e-481e-987c-fb1af73e4257\") " pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:12.278759 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.278299 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/60463747-bcd4-4066-9b4b-09ce3cf37a1f-ovnkube-script-lib\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.279405 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.279299 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/60463747-bcd4-4066-9b4b-09ce3cf37a1f-ovn-node-metrics-cert\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.280742 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.280719 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/145be0a6-160e-481e-987c-fb1af73e4257-agent-certs\") pod \"konnectivity-agent-8t9lx\" (UID: \"145be0a6-160e-481e-987c-fb1af73e4257\") " pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:12.284889 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.284836 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 23:48:12.290006 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.289944 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ssw9\" (UniqueName: \"kubernetes.io/projected/673c0dc9-eae9-4d18-ac43-825856e88954-kube-api-access-5ssw9\") pod \"node-resolver-sm2dg\" (UID: \"673c0dc9-eae9-4d18-ac43-825856e88954\") " pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.291362 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.291331 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4rhw\" (UniqueName: \"kubernetes.io/projected/d751b2b6-b78f-41b3-b895-5d559e9926d1-kube-api-access-x4rhw\") pod \"node-ca-j9bq7\" (UID: \"d751b2b6-b78f-41b3-b895-5d559e9926d1\") " pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.291649 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.291628 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg88l\" (UniqueName: \"kubernetes.io/projected/60463747-bcd4-4066-9b4b-09ce3cf37a1f-kube-api-access-vg88l\") pod \"ovnkube-node-j7xl5\" (UID: \"60463747-bcd4-4066-9b4b-09ce3cf37a1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.291725 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.291648 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxh5g\" (UniqueName: \"kubernetes.io/projected/5e0b8500-f7ec-40ad-885d-e358f1195532-kube-api-access-nxh5g\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:12.295330 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.292770 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq8jx\" (UniqueName: \"kubernetes.io/projected/341bdb2a-627b-4ea0-8236-5bf3ce3295a2-kube-api-access-mq8jx\") pod \"multus-additional-cni-plugins-zkdd7\" (UID: \"341bdb2a-627b-4ea0-8236-5bf3ce3295a2\") " pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.368160 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.368124 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rl9tb" Jul 06 23:48:12.377365 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.377328 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ssgq9" Jul 06 23:48:12.380609 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:12.380576 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0834213_feba_4bf3_b79d_cffe935e7be6.slice/crio-57d32e4f9150864274c9948e9473c4a3a4954dcbf52b1a967d06ffbbe7af97cf WatchSource:0}: Error finding container 57d32e4f9150864274c9948e9473c4a3a4954dcbf52b1a967d06ffbbe7af97cf: Status 404 returned error can't find the container with id 57d32e4f9150864274c9948e9473c4a3a4954dcbf52b1a967d06ffbbe7af97cf Jul 06 23:48:12.386053 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:12.386025 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4742585e_894a_425f_b770_15e50d8a2d82.slice/crio-5470158b9e96a2b918d322d7957ab578eb3f943f3f597e22ca683735bdf07909 WatchSource:0}: Error finding container 5470158b9e96a2b918d322d7957ab578eb3f943f3f597e22ca683735bdf07909: Status 404 returned error can't find the container with id 5470158b9e96a2b918d322d7957ab578eb3f943f3f597e22ca683735bdf07909 Jul 06 23:48:12.390754 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.390730 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" Jul 06 23:48:12.396895 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.396874 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" Jul 06 23:48:12.399558 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:12.399531 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d0ce9e_14d3_4d97_a170_dbb4b2772235.slice/crio-ae2f9fea08abdae0ba92704c6910114b98c9d76bdc9a23a475499a045cd0f308 WatchSource:0}: Error finding container ae2f9fea08abdae0ba92704c6910114b98c9d76bdc9a23a475499a045cd0f308: Status 404 returned error can't find the container with id ae2f9fea08abdae0ba92704c6910114b98c9d76bdc9a23a475499a045cd0f308 Jul 06 23:48:12.405662 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:12.405635 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb52deb_1807_4c14_ab9c_a19db334eff1.slice/crio-dcefca5151690b6dee649d0f0a8927694cb910b76eefc72df70e570340cb596c WatchSource:0}: Error finding container dcefca5151690b6dee649d0f0a8927694cb910b76eefc72df70e570340cb596c: Status 404 returned error can't find the container with id dcefca5151690b6dee649d0f0a8927694cb910b76eefc72df70e570340cb596c Jul 06 23:48:12.407279 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.407262 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j9bq7" Jul 06 23:48:12.416384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.416327 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:12.426387 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.426360 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:12.435231 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.435207 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" Jul 06 23:48:12.442827 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.442808 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sm2dg" Jul 06 23:48:12.780767 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.780680 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:12.780767 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.780732 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:12.780767 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:12.780756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:12.781050 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.780859 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:12.781050 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.780874 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:12.781050 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.780907 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:12.781050 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.780912 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs podName:5e0b8500-f7ec-40ad-885d-e358f1195532 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:13.780897454 +0000 UTC m=+4.043569634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs") pod "network-metrics-daemon-nhbt9" (UID: "5e0b8500-f7ec-40ad-885d-e358f1195532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:12.781050 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.780921 2583 projected.go:196] Error preparing data for projected volume kube-api-access-hxbgj for pod openshift-network-diagnostics/network-check-target-sfsws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:12.781050 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.780879 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:12.781050 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.780993 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj podName:5b6424b2-15c0-472d-8e25-4e69aab7538a nodeName:}" failed. No retries permitted until 2026-07-06 23:48:13.780957348 +0000 UTC m=+4.043629546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxbgj" (UniqueName: "kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj") pod "network-check-target-sfsws" (UID: "5b6424b2-15c0-472d-8e25-4e69aab7538a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:12.781050 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:12.781027 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret podName:9d9114c9-cf23-440f-a966-794fe592ced5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:13.781005936 +0000 UTC m=+4.043678114 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret") pod "global-pull-secret-syncer-vcbhn" (UID: "9d9114c9-cf23-440f-a966-794fe592ced5") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:13.102959 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.102826 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:12 +0000 UTC" deadline="2028-04-06 00:20:30.104128054 +0000 UTC" Jul 06 23:48:13.102959 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.102895 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15336h32m17.001237188s" Jul 06 23:48:13.175751 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.175720 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:13.175901 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:13.175878 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:13.186512 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.186407 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" event={"ID":"cfb52deb-1807-4c14-ab9c-a19db334eff1","Type":"ContainerStarted","Data":"dcefca5151690b6dee649d0f0a8927694cb910b76eefc72df70e570340cb596c"} Jul 06 23:48:13.187822 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.187785 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" event={"ID":"44d0ce9e-14d3-4d97-a170-dbb4b2772235","Type":"ContainerStarted","Data":"ae2f9fea08abdae0ba92704c6910114b98c9d76bdc9a23a475499a045cd0f308"} Jul 06 23:48:13.189465 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.189439 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ssgq9" event={"ID":"4742585e-894a-425f-b770-15e50d8a2d82","Type":"ContainerStarted","Data":"5470158b9e96a2b918d322d7957ab578eb3f943f3f597e22ca683735bdf07909"} Jul 06 23:48:13.190685 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.190663 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rl9tb" event={"ID":"f0834213-feba-4bf3-b79d-cffe935e7be6","Type":"ContainerStarted","Data":"57d32e4f9150864274c9948e9473c4a3a4954dcbf52b1a967d06ffbbe7af97cf"} Jul 06 23:48:13.750049 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:13.750013 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd751b2b6_b78f_41b3_b895_5d559e9926d1.slice/crio-a004a188a577cf2e297ba82e5c26d3743166ed806297a208473893607a52750e WatchSource:0}: Error finding container a004a188a577cf2e297ba82e5c26d3743166ed806297a208473893607a52750e: Status 404 returned error can't find the container with id a004a188a577cf2e297ba82e5c26d3743166ed806297a208473893607a52750e Jul 06 23:48:13.766271 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:13.766224 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod673c0dc9_eae9_4d18_ac43_825856e88954.slice/crio-3e6a47055ebcea8b9a925a936ed481a719fdc836571642eb72cb0f4ed0cf4a7b WatchSource:0}: Error finding container 3e6a47055ebcea8b9a925a936ed481a719fdc836571642eb72cb0f4ed0cf4a7b: Status 404 returned error can't find the container with id 3e6a47055ebcea8b9a925a936ed481a719fdc836571642eb72cb0f4ed0cf4a7b Jul 06 23:48:13.788799 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:13.788765 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341bdb2a_627b_4ea0_8236_5bf3ce3295a2.slice/crio-d3a0ab6fd022baab5fcdb834b7760981cc2fda2d38a16c1ad8500f233a1d7c85 WatchSource:0}: Error finding container d3a0ab6fd022baab5fcdb834b7760981cc2fda2d38a16c1ad8500f233a1d7c85: Status 404 returned error can't find the container with id d3a0ab6fd022baab5fcdb834b7760981cc2fda2d38a16c1ad8500f233a1d7c85 Jul 06 23:48:13.790209 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.790173 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:13.790373 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.790247 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:13.790373 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:13.790276 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:13.790488 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:13.790406 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:13.790488 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:13.790464 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs podName:5e0b8500-f7ec-40ad-885d-e358f1195532 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:15.790444349 +0000 UTC m=+6.053116531 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs") pod "network-metrics-daemon-nhbt9" (UID: "5e0b8500-f7ec-40ad-885d-e358f1195532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:13.790625 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:13.790602 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:13.790625 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:13.790625 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:13.790742 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:13.790638 2583 projected.go:196] Error preparing data for projected volume kube-api-access-hxbgj for pod openshift-network-diagnostics/network-check-target-sfsws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:13.790742 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:13.790659 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:13.790742 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:13.790700 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj podName:5b6424b2-15c0-472d-8e25-4e69aab7538a nodeName:}" failed. No retries permitted until 2026-07-06 23:48:15.790670119 +0000 UTC m=+6.053342300 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxbgj" (UniqueName: "kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj") pod "network-check-target-sfsws" (UID: "5b6424b2-15c0-472d-8e25-4e69aab7538a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:13.790742 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:13.790728 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret podName:9d9114c9-cf23-440f-a966-794fe592ced5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:15.790709762 +0000 UTC m=+6.053381940 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret") pod "global-pull-secret-syncer-vcbhn" (UID: "9d9114c9-cf23-440f-a966-794fe592ced5") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:13.792592 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:48:13.792547 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60463747_bcd4_4066_9b4b_09ce3cf37a1f.slice/crio-8c86eb1ba379cd4e4e1b30b676cf03468cd0c1f99ce9a3dfb4bbcb45e2dd9589 WatchSource:0}: Error finding container 8c86eb1ba379cd4e4e1b30b676cf03468cd0c1f99ce9a3dfb4bbcb45e2dd9589: Status 404 returned error can't find the container with id 8c86eb1ba379cd4e4e1b30b676cf03468cd0c1f99ce9a3dfb4bbcb45e2dd9589 Jul 06 23:48:14.103863 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.103528 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 23:43:12 +0000 UTC" deadline="2028-01-28 03:26:45.763510194 +0000 UTC" Jul 06 23:48:14.103863 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.103786 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13683h38m31.659730134s" Jul 06 23:48:14.177278 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.176681 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:14.177278 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:14.176819 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:14.177278 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.177132 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:14.177278 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:14.177235 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:14.194148 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.194112 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8t9lx" event={"ID":"145be0a6-160e-481e-987c-fb1af73e4257","Type":"ContainerStarted","Data":"0aaccddb354b8422726325142f5b7f06e8ca1a8035821ed3f25d0920c76e816c"} Jul 06 23:48:14.196332 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.196267 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sm2dg" event={"ID":"673c0dc9-eae9-4d18-ac43-825856e88954","Type":"ContainerStarted","Data":"3e6a47055ebcea8b9a925a936ed481a719fdc836571642eb72cb0f4ed0cf4a7b"} Jul 06 23:48:14.200525 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.200438 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j9bq7" event={"ID":"d751b2b6-b78f-41b3-b895-5d559e9926d1","Type":"ContainerStarted","Data":"a004a188a577cf2e297ba82e5c26d3743166ed806297a208473893607a52750e"} Jul 06 23:48:14.202206 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.202173 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" event={"ID":"60463747-bcd4-4066-9b4b-09ce3cf37a1f","Type":"ContainerStarted","Data":"8c86eb1ba379cd4e4e1b30b676cf03468cd0c1f99ce9a3dfb4bbcb45e2dd9589"} Jul 06 23:48:14.205078 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.205050 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" event={"ID":"341bdb2a-627b-4ea0-8236-5bf3ce3295a2","Type":"ContainerStarted","Data":"d3a0ab6fd022baab5fcdb834b7760981cc2fda2d38a16c1ad8500f233a1d7c85"} Jul 06 23:48:14.209659 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.209331 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal" event={"ID":"8ad2d9a7286370db5ed9312271a4a122","Type":"ContainerStarted","Data":"b06daadd6befb5bc934d7688469ba8aa204eb606d062007bb2118143565f9147"} Jul 06 23:48:14.225395 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:14.225297 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-188.ec2.internal" podStartSLOduration=2.225273923 podStartE2EDuration="2.225273923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:14.224845474 +0000 UTC m=+4.487517673" watchObservedRunningTime="2026-07-06 23:48:14.225273923 +0000 UTC m=+4.487946125" Jul 06 23:48:15.174888 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:15.174854 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:15.175362 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:15.175008 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:15.219436 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:15.219396 2583 generic.go:356] "Generic (PLEG): container finished" podID="769b25d8e28cec488cb47ae7f8f5c1fc" containerID="f58dd5ec8e31fd314926edc2dde2a4f845b4d0f3bac2ce864f7499da42fe887d" exitCode=0 Jul 06 23:48:15.220006 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:15.219977 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" event={"ID":"769b25d8e28cec488cb47ae7f8f5c1fc","Type":"ContainerDied","Data":"f58dd5ec8e31fd314926edc2dde2a4f845b4d0f3bac2ce864f7499da42fe887d"} Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:15.808034 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:15.808110 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:15.808143 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:15.808292 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:15.808358 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs podName:5e0b8500-f7ec-40ad-885d-e358f1195532 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:19.808337759 +0000 UTC m=+10.071009954 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs") pod "network-metrics-daemon-nhbt9" (UID: "5e0b8500-f7ec-40ad-885d-e358f1195532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:15.808819 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:15.808839 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:15.808852 2583 projected.go:196] Error preparing data for projected volume kube-api-access-hxbgj for pod openshift-network-diagnostics/network-check-target-sfsws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:15.808900 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj podName:5b6424b2-15c0-472d-8e25-4e69aab7538a nodeName:}" failed. No retries permitted until 2026-07-06 23:48:19.808884099 +0000 UTC m=+10.071556289 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxbgj" (UniqueName: "kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj") pod "network-check-target-sfsws" (UID: "5b6424b2-15c0-472d-8e25-4e69aab7538a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:15.808962 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:15.809033 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:15.808997 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret podName:9d9114c9-cf23-440f-a966-794fe592ced5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:19.808985567 +0000 UTC m=+10.071657749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret") pod "global-pull-secret-syncer-vcbhn" (UID: "9d9114c9-cf23-440f-a966-794fe592ced5") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:16.177437 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:16.177349 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:16.177966 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:16.177507 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:16.178053 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:16.178020 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:16.178206 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:16.178123 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:17.174835 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:17.174794 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:17.175043 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:17.174960 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:18.175056 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:18.175006 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:18.175736 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:18.175152 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:18.175736 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:18.175333 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:18.175736 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:18.175429 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:19.175109 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:19.175075 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:19.175644 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:19.175263 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:19.846421 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:19.846368 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:19.846614 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:19.846429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:19.846614 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:19.846512 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:19.846741 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:19.846650 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:19.846741 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:19.846671 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:19.846741 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:19.846681 2583 projected.go:196] Error preparing data for projected volume kube-api-access-hxbgj for pod openshift-network-diagnostics/network-check-target-sfsws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:19.846741 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:19.846728 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj podName:5b6424b2-15c0-472d-8e25-4e69aab7538a nodeName:}" failed. No retries permitted until 2026-07-06 23:48:27.846713742 +0000 UTC m=+18.109385916 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxbgj" (UniqueName: "kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj") pod "network-check-target-sfsws" (UID: "5b6424b2-15c0-472d-8e25-4e69aab7538a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:19.846953 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:19.846782 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:19.846953 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:19.846804 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret podName:9d9114c9-cf23-440f-a966-794fe592ced5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:27.846797481 +0000 UTC m=+18.109469657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret") pod "global-pull-secret-syncer-vcbhn" (UID: "9d9114c9-cf23-440f-a966-794fe592ced5") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:19.846953 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:19.846847 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:19.846953 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:19.846872 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs podName:5e0b8500-f7ec-40ad-885d-e358f1195532 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:27.846861866 +0000 UTC m=+18.109534040 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs") pod "network-metrics-daemon-nhbt9" (UID: "5e0b8500-f7ec-40ad-885d-e358f1195532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:20.176667 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:20.176585 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:20.177960 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:20.177757 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:20.177960 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:20.177841 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:20.177960 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:20.177928 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:21.175506 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:21.174982 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:21.175506 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:21.175139 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:22.174989 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:22.174949 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:22.175445 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:22.174999 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:22.175445 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:22.175088 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:22.175445 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:22.175208 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:23.175062 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:23.175032 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:23.175468 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:23.175173 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:24.175806 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:24.175763 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:24.176264 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:24.175902 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:24.176264 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:24.175979 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:24.176264 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:24.176103 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:25.175612 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:25.175578 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:25.175825 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:25.175701 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:26.175893 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:26.175864 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:26.175893 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:26.175903 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:26.176302 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:26.175993 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:26.176302 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:26.176111 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:27.175422 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:27.175387 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:27.175636 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:27.175516 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:27.908526 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:27.908493 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:27.908543 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:27.908665 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:27.908684 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:27.908705 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:27.908718 2583 projected.go:196] Error preparing data for projected volume kube-api-access-hxbgj for pod openshift-network-diagnostics/network-check-target-sfsws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:27.908784 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj podName:5b6424b2-15c0-472d-8e25-4e69aab7538a nodeName:}" failed. No retries permitted until 2026-07-06 23:48:43.908764263 +0000 UTC m=+34.171436460 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxbgj" (UniqueName: "kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj") pod "network-check-target-sfsws" (UID: "5b6424b2-15c0-472d-8e25-4e69aab7538a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:27.908785 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:27.908839 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret podName:9d9114c9-cf23-440f-a966-794fe592ced5 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:43.908826777 +0000 UTC m=+34.171498950 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret") pod "global-pull-secret-syncer-vcbhn" (UID: "9d9114c9-cf23-440f-a966-794fe592ced5") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:27.908789 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:27.909032 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:27.908878 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs podName:5e0b8500-f7ec-40ad-885d-e358f1195532 nodeName:}" failed. No retries permitted until 2026-07-06 23:48:43.908867103 +0000 UTC m=+34.171539277 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs") pod "network-metrics-daemon-nhbt9" (UID: "5e0b8500-f7ec-40ad-885d-e358f1195532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:28.175401 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:28.175325 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:28.175585 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:28.175462 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:28.175585 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:28.175522 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:28.175691 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:28.175628 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:29.175092 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:29.175059 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:29.175469 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:29.175205 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:30.176131 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:30.176078 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:30.176603 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:30.176194 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:30.176603 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:30.176241 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:30.176603 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:30.176323 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:31.175121 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:31.175075 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:31.175293 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:31.175212 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:32.178074 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.177638 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:32.178710 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.177675 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:32.178710 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:32.178205 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:32.178710 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:32.178239 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:32.254031 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.254006 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" event={"ID":"769b25d8e28cec488cb47ae7f8f5c1fc","Type":"ContainerStarted","Data":"6635de5a39eaacd0a4734c025d16fe16ea3108a66fdeb184e3fe77bce81a2630"} Jul 06 23:48:32.255419 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.255398 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" event={"ID":"cfb52deb-1807-4c14-ab9c-a19db334eff1","Type":"ContainerStarted","Data":"930869563de756a5a9d964991b37ce9e2b5870b73c35cb3aae9192c5f05b906f"} Jul 06 23:48:32.256746 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.256724 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" event={"ID":"44d0ce9e-14d3-4d97-a170-dbb4b2772235","Type":"ContainerStarted","Data":"be4207aca0f1a766132dd2a9aea93aa71c3d3f196d1f99832b5e23910920e648"} Jul 06 23:48:32.258207 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.258188 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ssgq9" event={"ID":"4742585e-894a-425f-b770-15e50d8a2d82","Type":"ContainerStarted","Data":"5d9df22c60b78b26516945ac2fa2bf1aa60002d735a7be4db1795d2b36b5c04e"} Jul 06 23:48:32.260788 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.260768 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" event={"ID":"60463747-bcd4-4066-9b4b-09ce3cf37a1f","Type":"ContainerStarted","Data":"2019a3baba2b342b6c1297a658b71027e96e1d35af5b4e9b795ceb2042852659"} Jul 06 23:48:32.260871 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.260827 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" event={"ID":"60463747-bcd4-4066-9b4b-09ce3cf37a1f","Type":"ContainerStarted","Data":"4ddfc07ddeeba956140ffca2704e43e99e22b82bfe4da47322632ac8b3c4c197"} Jul 06 23:48:32.260914 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.260871 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" event={"ID":"60463747-bcd4-4066-9b4b-09ce3cf37a1f","Type":"ContainerStarted","Data":"1b2462d70d3ea7f958df9f3e7516495a29efaa7eae3383780369bbffde55a752"} Jul 06 23:48:32.260914 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.260908 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" event={"ID":"60463747-bcd4-4066-9b4b-09ce3cf37a1f","Type":"ContainerStarted","Data":"fdf81de1c247003caa2d04b6b08131de494186b0a247775445859b621c5d9559"} Jul 06 23:48:32.260986 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.260950 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" event={"ID":"60463747-bcd4-4066-9b4b-09ce3cf37a1f","Type":"ContainerStarted","Data":"ba454931b789558870dcda8cac3a79e4a08292049577b1a5845f1f1049bfba6e"} Jul 06 23:48:32.262333 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.262315 2583 generic.go:356] "Generic (PLEG): container finished" podID="341bdb2a-627b-4ea0-8236-5bf3ce3295a2" containerID="4ed0f4711a3f29624b53a187d7a42c6eb7613fcf7afb3b37261bd3317cf9834d" exitCode=0 Jul 06 23:48:32.262417 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.262380 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" event={"ID":"341bdb2a-627b-4ea0-8236-5bf3ce3295a2","Type":"ContainerDied","Data":"4ed0f4711a3f29624b53a187d7a42c6eb7613fcf7afb3b37261bd3317cf9834d"} Jul 06 23:48:32.263982 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.263959 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8t9lx" event={"ID":"145be0a6-160e-481e-987c-fb1af73e4257","Type":"ContainerStarted","Data":"32adbf7c649dd2e617cc53f91faf7b8d0cbdf9f9b9c81737afa51b261a37539a"} Jul 06 23:48:32.265576 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.265549 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sm2dg" event={"ID":"673c0dc9-eae9-4d18-ac43-825856e88954","Type":"ContainerStarted","Data":"c14d27059bb34caf6ed8577c7746d89d15e2fba2e66445787c4f231a9ebe0f7b"} Jul 06 23:48:32.266925 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.266902 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j9bq7" event={"ID":"d751b2b6-b78f-41b3-b895-5d559e9926d1","Type":"ContainerStarted","Data":"54da70d4fb4b32b5113b27ba48631b52c68e63f1c086c34988bd3f6e8e156ab2"} Jul 06 23:48:32.286100 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.286026 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-188.ec2.internal" podStartSLOduration=20.286013192 podStartE2EDuration="20.286013192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:32.26845865 +0000 UTC m=+22.531130844" watchObservedRunningTime="2026-07-06 23:48:32.286013192 +0000 UTC m=+22.548685386" Jul 06 23:48:32.286100 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.286089 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ssgq9" podStartSLOduration=3.118922352 podStartE2EDuration="22.286085292s" totalImagesPullingTime="19.16716294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:32.285802036 +0000 UTC m=+22.548474232" watchObservedRunningTime="2026-07-06 23:48:32.286085292 +0000 UTC m=+22.548757486" Jul 06 23:48:32.304009 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.303966 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nkgkq" podStartSLOduration=3.175597863 podStartE2EDuration="22.303951997s" totalImagesPullingTime="19.128354134s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:32.303578523 +0000 UTC m=+22.566250717" watchObservedRunningTime="2026-07-06 23:48:32.303951997 +0000 UTC m=+22.566624192" Jul 06 23:48:32.318315 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.318277 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j9bq7" podStartSLOduration=3.54034414 podStartE2EDuration="21.318263752s" totalImagesPullingTime="17.777919612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:32.317908574 +0000 UTC m=+22.580580773" watchObservedRunningTime="2026-07-06 23:48:32.318263752 +0000 UTC m=+22.580935946" Jul 06 23:48:32.332512 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.332451 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8t9lx" podStartSLOduration=3.598990671 podStartE2EDuration="21.33243726s" totalImagesPullingTime="17.733446589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:32.331873163 +0000 UTC m=+22.594545357" watchObservedRunningTime="2026-07-06 23:48:32.33243726 +0000 UTC m=+22.595109455" Jul 06 23:48:32.372272 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:32.372230 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sm2dg" podStartSLOduration=3.610874507 podStartE2EDuration="21.372215007s" totalImagesPullingTime="17.7613405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:32.371867273 +0000 UTC m=+22.634539479" watchObservedRunningTime="2026-07-06 23:48:32.372215007 +0000 UTC m=+22.634887201" Jul 06 23:48:33.175516 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:33.175466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:33.175716 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:33.175636 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:33.200054 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:33.200029 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 23:48:33.273578 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:33.273540 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" event={"ID":"60463747-bcd4-4066-9b4b-09ce3cf37a1f","Type":"ContainerStarted","Data":"e4408aa98e101ecff53e574f9b604b8a1a76da52962317e7515e49110b67819d"} Jul 06 23:48:33.275482 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:33.275447 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" event={"ID":"cfb52deb-1807-4c14-ab9c-a19db334eff1","Type":"ContainerStarted","Data":"b67c7ec4562ab552398674dbfb7d1d2a468ea2c2c11e3f3524291aa0ce84a15f"} Jul 06 23:48:33.277268 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:33.277242 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rl9tb" event={"ID":"f0834213-feba-4bf3-b79d-cffe935e7be6","Type":"ContainerStarted","Data":"292270cd1708921e67863ebe4eb9dcfaace2de1d740251c0202d9560ed3cbd0f"} Jul 06 23:48:33.292018 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:33.291976 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rl9tb" podStartSLOduration=17.207141281 podStartE2EDuration="23.29196165s" totalImagesPullingTime="6.084820369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:33.291672723 +0000 UTC m=+23.554344914" watchObservedRunningTime="2026-07-06 23:48:33.29196165 +0000 UTC m=+23.554633844" Jul 06 23:48:34.099208 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.099177 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:34.099680 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.099664 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:34.134841 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.134742 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T23:48:33.200047884Z","UUID":"4736de0a-d160-46dc-94df-31194712d47a","Handler":null,"Name":"","Endpoint":""} Jul 06 23:48:34.136395 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.136372 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 23:48:34.136502 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.136403 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 23:48:34.178379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.178342 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:34.178590 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:34.178452 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:34.178590 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.178500 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:34.178706 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:34.178613 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:34.281553 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.281507 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" event={"ID":"cfb52deb-1807-4c14-ab9c-a19db334eff1","Type":"ContainerStarted","Data":"5d8adb18499a9ac754f89ce24b77a1a09e76e893d87aaa488743472a69c772f7"} Jul 06 23:48:34.282018 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.281753 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:34.282373 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.282351 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8t9lx" Jul 06 23:48:34.300636 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:34.300594 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h566q" podStartSLOduration=3.067124636 podStartE2EDuration="24.300578951s" totalImagesPullingTime="21.233454315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:34.300122819 +0000 UTC m=+24.562795031" watchObservedRunningTime="2026-07-06 23:48:34.300578951 +0000 UTC m=+24.563251145" Jul 06 23:48:35.175014 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:35.174822 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:35.175199 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:35.175118 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:35.288108 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:35.288073 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" event={"ID":"60463747-bcd4-4066-9b4b-09ce3cf37a1f","Type":"ContainerStarted","Data":"cbfe1f8e6d4b01654c1af5b550e1d9cc3baff0bfa862bf343f78a10da0435219"} Jul 06 23:48:36.178754 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:36.178728 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:36.178908 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:36.178826 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:36.178908 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:36.178853 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:36.179049 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:36.178918 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:37.175125 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:37.175091 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:37.175767 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:37.175216 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:38.178388 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:38.178209 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:38.178914 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:38.178267 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:38.178914 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:38.178465 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:38.178914 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:38.178544 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:38.298421 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:38.298386 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" event={"ID":"60463747-bcd4-4066-9b4b-09ce3cf37a1f","Type":"ContainerStarted","Data":"62ac5d8796859d626bc3f797eb146f4cc54af61f26b3145ae8908c37cebb966e"} Jul 06 23:48:38.298696 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:38.298672 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:38.300277 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:38.300257 2583 generic.go:356] "Generic (PLEG): container finished" podID="341bdb2a-627b-4ea0-8236-5bf3ce3295a2" containerID="055a9b426f2d0c77dc05a409fa664d39c102eb1051d56a318f8553ffe75895f0" exitCode=0 Jul 06 23:48:38.300376 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:38.300301 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" event={"ID":"341bdb2a-627b-4ea0-8236-5bf3ce3295a2","Type":"ContainerDied","Data":"055a9b426f2d0c77dc05a409fa664d39c102eb1051d56a318f8553ffe75895f0"} Jul 06 23:48:38.314718 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:38.314700 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:38.335989 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:38.335944 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" podStartSLOduration=9.429424437 podStartE2EDuration="27.335931669s" totalImagesPullingTime="17.906507232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:38.335873755 +0000 UTC m=+28.598545949" watchObservedRunningTime="2026-07-06 23:48:38.335931669 +0000 UTC m=+28.598603863" Jul 06 23:48:39.175725 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.175694 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:39.175955 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:39.175815 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:39.303089 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.303058 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:39.303591 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.303143 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:39.318082 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.318055 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:48:39.942634 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.942422 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcbhn"] Jul 06 23:48:39.942820 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.942737 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:39.942884 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:39.942847 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:39.945860 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.945825 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nhbt9"] Jul 06 23:48:39.945983 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.945915 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:39.946030 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:39.946001 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:39.957383 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.957359 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sfsws"] Jul 06 23:48:39.957491 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:39.957458 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:39.957579 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:39.957560 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:40.307230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:40.307187 2583 generic.go:356] "Generic (PLEG): container finished" podID="341bdb2a-627b-4ea0-8236-5bf3ce3295a2" containerID="bc9d1e7b66697d7ece28c284042c19e74319fa4408278df411c0cb7340e361e1" exitCode=0 Jul 06 23:48:40.307620 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:40.307263 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" event={"ID":"341bdb2a-627b-4ea0-8236-5bf3ce3295a2","Type":"ContainerDied","Data":"bc9d1e7b66697d7ece28c284042c19e74319fa4408278df411c0cb7340e361e1"} Jul 06 23:48:41.175022 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:41.174945 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:41.175147 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:41.174945 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:41.175147 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:41.175050 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:41.175221 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:41.175144 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:41.311996 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:41.311964 2583 generic.go:356] "Generic (PLEG): container finished" podID="341bdb2a-627b-4ea0-8236-5bf3ce3295a2" containerID="1d199dfb6745c7bddc08a43ec1750d0f5ba5d9fe185a7aa7516ad0a33ab78a98" exitCode=0 Jul 06 23:48:41.312688 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:41.312053 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" event={"ID":"341bdb2a-627b-4ea0-8236-5bf3ce3295a2","Type":"ContainerDied","Data":"1d199dfb6745c7bddc08a43ec1750d0f5ba5d9fe185a7aa7516ad0a33ab78a98"} Jul 06 23:48:41.550737 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:41.550706 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sm2dg_673c0dc9-eae9-4d18-ac43-825856e88954/dns-node-resolver/0.log" Jul 06 23:48:42.178763 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:42.178736 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:42.178931 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:42.178846 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:42.510675 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:42.510650 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j9bq7_d751b2b6-b78f-41b3-b895-5d559e9926d1/node-ca/0.log" Jul 06 23:48:43.175418 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:43.175385 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:43.175588 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:43.175426 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:43.175588 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.175508 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:43.175668 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.175624 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:43.933617 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:43.933572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:43.933617 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:43.933619 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:43.934261 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:43.933665 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:43.934261 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.933741 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:43.934261 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.933780 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 23:48:43.934261 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.933788 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:43.934261 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.933798 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 23:48:43.934261 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.933816 2583 projected.go:196] Error preparing data for projected volume kube-api-access-hxbgj for pod openshift-network-diagnostics/network-check-target-sfsws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:43.934261 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.933823 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret podName:9d9114c9-cf23-440f-a966-794fe592ced5 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:15.933801533 +0000 UTC m=+66.196473712 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret") pod "global-pull-secret-syncer-vcbhn" (UID: "9d9114c9-cf23-440f-a966-794fe592ced5") : object "kube-system"/"original-pull-secret" not registered Jul 06 23:48:43.934261 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.933850 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs podName:5e0b8500-f7ec-40ad-885d-e358f1195532 nodeName:}" failed. No retries permitted until 2026-07-06 23:49:15.933830564 +0000 UTC m=+66.196502740 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs") pod "network-metrics-daemon-nhbt9" (UID: "5e0b8500-f7ec-40ad-885d-e358f1195532") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 23:48:43.934261 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:43.933874 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj podName:5b6424b2-15c0-472d-8e25-4e69aab7538a nodeName:}" failed. No retries permitted until 2026-07-06 23:49:15.933862317 +0000 UTC m=+66.196534493 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hxbgj" (UniqueName: "kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj") pod "network-check-target-sfsws" (UID: "5b6424b2-15c0-472d-8e25-4e69aab7538a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 23:48:44.175906 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:44.175873 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:44.176079 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:44.176014 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:45.175823 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:45.175788 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:45.176257 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:45.175919 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:45.176257 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:45.175976 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:45.176257 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:45.176078 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:46.177689 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:46.177619 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:46.178051 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:46.177720 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:46.327241 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:46.327209 2583 generic.go:356] "Generic (PLEG): container finished" podID="341bdb2a-627b-4ea0-8236-5bf3ce3295a2" containerID="fe2e39b2e720deac7d86afc38aba733265f0719c39471ff78eb2ffcd5a5d3575" exitCode=0 Jul 06 23:48:46.327426 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:46.327273 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" event={"ID":"341bdb2a-627b-4ea0-8236-5bf3ce3295a2","Type":"ContainerDied","Data":"fe2e39b2e720deac7d86afc38aba733265f0719c39471ff78eb2ffcd5a5d3575"} Jul 06 23:48:47.175295 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:47.175107 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:47.175491 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:47.175139 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:47.175491 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:47.175402 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:47.175618 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:47.175497 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:47.333035 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:47.333000 2583 generic.go:356] "Generic (PLEG): container finished" podID="341bdb2a-627b-4ea0-8236-5bf3ce3295a2" containerID="2cc0329a4ec5df0917c05cae8f596021452701a360bdcf30ae17443866479e1e" exitCode=0 Jul 06 23:48:47.333390 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:47.333062 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" event={"ID":"341bdb2a-627b-4ea0-8236-5bf3ce3295a2","Type":"ContainerDied","Data":"2cc0329a4ec5df0917c05cae8f596021452701a360bdcf30ae17443866479e1e"} Jul 06 23:48:48.175357 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:48.175323 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:48.175569 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:48.175446 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:48.338659 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:48.338630 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" event={"ID":"341bdb2a-627b-4ea0-8236-5bf3ce3295a2","Type":"ContainerStarted","Data":"d76d290e63d01f70f1ccad32a581a40b8cf3a0dfaa06b0bac99a82d1a3f3ae4a"} Jul 06 23:48:48.366552 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:48.366508 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zkdd7" podStartSLOduration=7.7503977509999995 podStartE2EDuration="37.366492063s" totalImagesPullingTime="29.616094312s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:48:48.366185094 +0000 UTC m=+38.628857279" watchObservedRunningTime="2026-07-06 23:48:48.366492063 +0000 UTC m=+38.629164248" Jul 06 23:48:49.175327 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:49.175292 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:49.175537 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:49.175395 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:49.175537 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:49.175447 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:49.175621 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:49.175551 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:50.175561 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:50.175522 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:50.175945 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:50.175615 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:51.175500 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:51.175464 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:51.175691 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:51.175586 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:51.175691 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:51.175637 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:51.176033 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:51.175717 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:52.175262 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:52.175233 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:52.175416 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:52.175331 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:53.174810 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:53.174777 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:53.175215 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:53.174781 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:53.175215 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:53.174874 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:53.175215 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:53.174968 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:54.175451 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:54.175417 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:54.175880 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:54.175566 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:55.175506 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:55.175461 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:55.175963 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:55.175461 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:55.175963 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:55.175589 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:55.175963 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:55.175647 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:56.175256 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:56.175213 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:56.175449 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:56.175322 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:57.175330 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:57.175293 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:57.175750 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:57.175293 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:57.175750 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:57.175422 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:57.175750 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:57.175460 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:48:58.175711 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:58.175681 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:48:58.176093 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:58.175782 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:48:59.175756 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:59.175723 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:48:59.176215 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:59.175848 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:48:59.176215 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:48:59.175893 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:48:59.176215 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:48:59.175982 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:49:00.176616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:00.176586 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:49:00.177009 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:49:00.176674 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:49:01.175398 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:01.175366 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:49:01.175593 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:49:01.175488 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:49:01.175593 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:01.175530 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:49:01.175711 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:49:01.175626 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:49:02.177929 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:02.177903 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:49:02.178339 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:49:02.178012 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:49:03.175433 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:03.175397 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:49:03.175669 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:49:03.175524 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:49:03.175669 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:03.175574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:49:03.175669 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:49:03.175662 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:49:04.175233 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:04.175200 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:49:04.175716 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:49:04.175295 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vcbhn" podUID="9d9114c9-cf23-440f-a966-794fe592ced5" Jul 06 23:49:05.175033 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.174998 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:49:05.175206 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.174998 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:49:05.175206 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:49:05.175103 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sfsws" podUID="5b6424b2-15c0-472d-8e25-4e69aab7538a" Jul 06 23:49:05.175206 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:49:05.175180 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nhbt9" podUID="5e0b8500-f7ec-40ad-885d-e358f1195532" Jul 06 23:49:05.566975 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.566950 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-188.ec2.internal" event="NodeReady" Jul 06 23:49:05.567134 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.567069 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 23:49:05.616180 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.616144 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-755c4fcb8f-pwv98"] Jul 06 23:49:05.617814 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.617800 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.620832 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.620811 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 23:49:05.620978 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.620962 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 23:49:05.621235 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.621221 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6sf8h\"" Jul 06 23:49:05.621496 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.621468 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 23:49:05.636452 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.636425 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5pg5w"] Jul 06 23:49:05.637920 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.637905 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pg5w" Jul 06 23:49:05.639767 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.639749 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gjb9z"] Jul 06 23:49:05.641457 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.641443 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.648854 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.648834 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-755c4fcb8f-pwv98"] Jul 06 23:49:05.651711 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.651690 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 23:49:05.655582 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.655557 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 23:49:05.655666 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.655590 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 23:49:05.655666 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.655661 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 23:49:05.656143 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.656125 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 23:49:05.656221 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.656211 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 23:49:05.656280 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.656242 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 23:49:05.656492 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.656463 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gz82q\"" Jul 06 23:49:05.656566 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.656552 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wqknp\"" Jul 06 23:49:05.658547 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.658529 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 23:49:05.659514 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.659493 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pg5w"] Jul 06 23:49:05.671566 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.671532 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gjb9z"] Jul 06 23:49:05.685066 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685041 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0350166e-a62f-49f8-911b-9f6f7870a562-image-registry-private-configuration\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.685170 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685070 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0350166e-a62f-49f8-911b-9f6f7870a562-registry-certificates\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.685170 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685091 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc7855c6-e0ea-4385-b08f-a91bfb641524-data-volume\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.685170 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685131 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfsx9\" (UniqueName: \"kubernetes.io/projected/dc7855c6-e0ea-4385-b08f-a91bfb641524-kube-api-access-zfsx9\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.685170 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685150 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0350166e-a62f-49f8-911b-9f6f7870a562-registry-tls\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.685170 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685165 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpwb2\" (UniqueName: \"kubernetes.io/projected/0812878f-d9b5-4456-8b62-9a020d4cc2a4-kube-api-access-gpwb2\") pod \"ingress-canary-5pg5w\" (UID: \"0812878f-d9b5-4456-8b62-9a020d4cc2a4\") " pod="openshift-ingress-canary/ingress-canary-5pg5w" Jul 06 23:49:05.685401 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685183 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0812878f-d9b5-4456-8b62-9a020d4cc2a4-cert\") pod \"ingress-canary-5pg5w\" (UID: \"0812878f-d9b5-4456-8b62-9a020d4cc2a4\") " pod="openshift-ingress-canary/ingress-canary-5pg5w" Jul 06 23:49:05.685401 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685233 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dc7855c6-e0ea-4385-b08f-a91bfb641524-crio-socket\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.685401 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685274 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0350166e-a62f-49f8-911b-9f6f7870a562-bound-sa-token\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.685401 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685310 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0350166e-a62f-49f8-911b-9f6f7870a562-ca-trust-extracted\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.685401 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685348 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dc7855c6-e0ea-4385-b08f-a91bfb641524-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.685401 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dc7855c6-e0ea-4385-b08f-a91bfb641524-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.685401 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685380 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6q4g\" (UniqueName: \"kubernetes.io/projected/0350166e-a62f-49f8-911b-9f6f7870a562-kube-api-access-z6q4g\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.685401 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685397 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0350166e-a62f-49f8-911b-9f6f7870a562-trusted-ca\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.685663 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.685412 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0350166e-a62f-49f8-911b-9f6f7870a562-installation-pull-secrets\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.726604 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.726573 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bs4cn"] Jul 06 23:49:05.728196 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.728180 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.733033 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.732983 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 23:49:05.733033 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.733012 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5f2wr\"" Jul 06 23:49:05.733213 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.733070 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 23:49:05.752046 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.752018 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bs4cn"] Jul 06 23:49:05.786417 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786383 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dc7855c6-e0ea-4385-b08f-a91bfb641524-crio-socket\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.786417 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786416 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0350166e-a62f-49f8-911b-9f6f7870a562-bound-sa-token\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.786650 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786444 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0350166e-a62f-49f8-911b-9f6f7870a562-ca-trust-extracted\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.786650 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786484 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dc7855c6-e0ea-4385-b08f-a91bfb641524-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.786650 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786509 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dc7855c6-e0ea-4385-b08f-a91bfb641524-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.786650 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6q4g\" (UniqueName: \"kubernetes.io/projected/0350166e-a62f-49f8-911b-9f6f7870a562-kube-api-access-z6q4g\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.786650 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786510 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dc7855c6-e0ea-4385-b08f-a91bfb641524-crio-socket\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.786650 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0350166e-a62f-49f8-911b-9f6f7870a562-trusted-ca\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.786650 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0350166e-a62f-49f8-911b-9f6f7870a562-installation-pull-secrets\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.786958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786683 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0350166e-a62f-49f8-911b-9f6f7870a562-image-registry-private-configuration\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.786958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786717 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0350166e-a62f-49f8-911b-9f6f7870a562-registry-certificates\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.786958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786752 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc7855c6-e0ea-4385-b08f-a91bfb641524-data-volume\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.786958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786784 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnjw7\" (UniqueName: \"kubernetes.io/projected/41c2146f-78d2-499c-81f3-ae76240f8c62-kube-api-access-dnjw7\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.786958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfsx9\" (UniqueName: \"kubernetes.io/projected/dc7855c6-e0ea-4385-b08f-a91bfb641524-kube-api-access-zfsx9\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.786958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786886 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0350166e-a62f-49f8-911b-9f6f7870a562-registry-tls\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.786958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786910 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpwb2\" (UniqueName: \"kubernetes.io/projected/0812878f-d9b5-4456-8b62-9a020d4cc2a4-kube-api-access-gpwb2\") pod \"ingress-canary-5pg5w\" (UID: \"0812878f-d9b5-4456-8b62-9a020d4cc2a4\") " pod="openshift-ingress-canary/ingress-canary-5pg5w" Jul 06 23:49:05.786958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786936 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/41c2146f-78d2-499c-81f3-ae76240f8c62-config-volume\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.786958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786958 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c2146f-78d2-499c-81f3-ae76240f8c62-metrics-tls\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.787398 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.786980 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c2146f-78d2-499c-81f3-ae76240f8c62-tmp-dir\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.787398 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.787011 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0812878f-d9b5-4456-8b62-9a020d4cc2a4-cert\") pod \"ingress-canary-5pg5w\" (UID: \"0812878f-d9b5-4456-8b62-9a020d4cc2a4\") " pod="openshift-ingress-canary/ingress-canary-5pg5w" Jul 06 23:49:05.787398 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.787035 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0350166e-a62f-49f8-911b-9f6f7870a562-ca-trust-extracted\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.787398 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.787256 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dc7855c6-e0ea-4385-b08f-a91bfb641524-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.787677 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.787658 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc7855c6-e0ea-4385-b08f-a91bfb641524-data-volume\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.787795 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.787760 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0350166e-a62f-49f8-911b-9f6f7870a562-trusted-ca\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.788110 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.788087 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0350166e-a62f-49f8-911b-9f6f7870a562-registry-certificates\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.790843 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.790822 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dc7855c6-e0ea-4385-b08f-a91bfb641524-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.790974 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.790955 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0812878f-d9b5-4456-8b62-9a020d4cc2a4-cert\") pod \"ingress-canary-5pg5w\" (UID: \"0812878f-d9b5-4456-8b62-9a020d4cc2a4\") " pod="openshift-ingress-canary/ingress-canary-5pg5w" Jul 06 23:49:05.791031 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.790976 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0350166e-a62f-49f8-911b-9f6f7870a562-installation-pull-secrets\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.791031 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.791017 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0350166e-a62f-49f8-911b-9f6f7870a562-registry-tls\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.791106 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.791062 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0350166e-a62f-49f8-911b-9f6f7870a562-image-registry-private-configuration\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.795918 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.795892 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfsx9\" (UniqueName: \"kubernetes.io/projected/dc7855c6-e0ea-4385-b08f-a91bfb641524-kube-api-access-zfsx9\") pod \"insights-runtime-extractor-gjb9z\" (UID: \"dc7855c6-e0ea-4385-b08f-a91bfb641524\") " pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:05.796394 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.796365 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0350166e-a62f-49f8-911b-9f6f7870a562-bound-sa-token\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.796679 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.796654 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpwb2\" (UniqueName: \"kubernetes.io/projected/0812878f-d9b5-4456-8b62-9a020d4cc2a4-kube-api-access-gpwb2\") pod \"ingress-canary-5pg5w\" (UID: \"0812878f-d9b5-4456-8b62-9a020d4cc2a4\") " pod="openshift-ingress-canary/ingress-canary-5pg5w" Jul 06 23:49:05.796864 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.796847 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6q4g\" (UniqueName: \"kubernetes.io/projected/0350166e-a62f-49f8-911b-9f6f7870a562-kube-api-access-z6q4g\") pod \"image-registry-755c4fcb8f-pwv98\" (UID: \"0350166e-a62f-49f8-911b-9f6f7870a562\") " pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.887990 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.887904 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnjw7\" (UniqueName: \"kubernetes.io/projected/41c2146f-78d2-499c-81f3-ae76240f8c62-kube-api-access-dnjw7\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.887990 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.887970 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/41c2146f-78d2-499c-81f3-ae76240f8c62-config-volume\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.888204 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.887994 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c2146f-78d2-499c-81f3-ae76240f8c62-metrics-tls\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.888204 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.888016 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c2146f-78d2-499c-81f3-ae76240f8c62-tmp-dir\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.888396 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.888378 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41c2146f-78d2-499c-81f3-ae76240f8c62-tmp-dir\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.888548 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.888529 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/41c2146f-78d2-499c-81f3-ae76240f8c62-config-volume\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.890299 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.890279 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/41c2146f-78d2-499c-81f3-ae76240f8c62-metrics-tls\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.898326 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.898303 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnjw7\" (UniqueName: \"kubernetes.io/projected/41c2146f-78d2-499c-81f3-ae76240f8c62-kube-api-access-dnjw7\") pod \"dns-default-bs4cn\" (UID: \"41c2146f-78d2-499c-81f3-ae76240f8c62\") " pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:05.929333 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.929306 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:05.946759 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.946739 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5pg5w" Jul 06 23:49:05.952452 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:05.952431 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gjb9z" Jul 06 23:49:06.037068 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.036275 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:06.115600 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.115549 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-755c4fcb8f-pwv98"] Jul 06 23:49:06.120170 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:06.120145 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0350166e_a62f_49f8_911b_9f6f7870a562.slice/crio-dc96ffde5a75f48f3a08a4123b33e74e019f564b756ecc04cb88f88f963835ab WatchSource:0}: Error finding container dc96ffde5a75f48f3a08a4123b33e74e019f564b756ecc04cb88f88f963835ab: Status 404 returned error can't find the container with id dc96ffde5a75f48f3a08a4123b33e74e019f564b756ecc04cb88f88f963835ab Jul 06 23:49:06.126645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.126624 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5pg5w"] Jul 06 23:49:06.133005 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:06.132974 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0812878f_d9b5_4456_8b62_9a020d4cc2a4.slice/crio-510cda19f3d1756580104397affa1a4447674c3dbbbf98c85c7479bf4a7d7cd0 WatchSource:0}: Error finding container 510cda19f3d1756580104397affa1a4447674c3dbbbf98c85c7479bf4a7d7cd0: Status 404 returned error can't find the container with id 510cda19f3d1756580104397affa1a4447674c3dbbbf98c85c7479bf4a7d7cd0 Jul 06 23:49:06.141440 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.141379 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gjb9z"] Jul 06 23:49:06.154781 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:06.154755 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc7855c6_e0ea_4385_b08f_a91bfb641524.slice/crio-0666820635dfdc21eca1e05613989d423e93dc08643cd7444cc5ca0cbb799af0 WatchSource:0}: Error finding container 0666820635dfdc21eca1e05613989d423e93dc08643cd7444cc5ca0cbb799af0: Status 404 returned error can't find the container with id 0666820635dfdc21eca1e05613989d423e93dc08643cd7444cc5ca0cbb799af0 Jul 06 23:49:06.175271 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.175245 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:49:06.180525 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.180502 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:49:06.183208 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.183187 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bs4cn"] Jul 06 23:49:06.188388 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:06.188234 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c2146f_78d2_499c_81f3_ae76240f8c62.slice/crio-cdb0018a17f4c67bf13dd8a64ee460e6aa404d8fe727ec891d86fbc681d24514 WatchSource:0}: Error finding container cdb0018a17f4c67bf13dd8a64ee460e6aa404d8fe727ec891d86fbc681d24514: Status 404 returned error can't find the container with id cdb0018a17f4c67bf13dd8a64ee460e6aa404d8fe727ec891d86fbc681d24514 Jul 06 23:49:06.188828 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.188811 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b"] Jul 06 23:49:06.191103 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.191090 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" Jul 06 23:49:06.192990 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.192967 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wfgfj\"" Jul 06 23:49:06.193242 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.193223 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 23:49:06.199762 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.199742 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b"] Jul 06 23:49:06.291453 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.291418 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/243e7b42-f551-47ad-a169-c0fc8df2ca2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-4p46b\" (UID: \"243e7b42-f551-47ad-a169-c0fc8df2ca2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" Jul 06 23:49:06.378805 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.378771 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bs4cn" event={"ID":"41c2146f-78d2-499c-81f3-ae76240f8c62","Type":"ContainerStarted","Data":"cdb0018a17f4c67bf13dd8a64ee460e6aa404d8fe727ec891d86fbc681d24514"} Jul 06 23:49:06.379848 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.379826 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pg5w" event={"ID":"0812878f-d9b5-4456-8b62-9a020d4cc2a4","Type":"ContainerStarted","Data":"510cda19f3d1756580104397affa1a4447674c3dbbbf98c85c7479bf4a7d7cd0"} Jul 06 23:49:06.381258 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.381230 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" event={"ID":"0350166e-a62f-49f8-911b-9f6f7870a562","Type":"ContainerStarted","Data":"b2d53bad3e956a95224805b65ab1846364f948ddddf3571bfda11115d082baf2"} Jul 06 23:49:06.381384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.381363 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:06.381443 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.381404 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" event={"ID":"0350166e-a62f-49f8-911b-9f6f7870a562","Type":"ContainerStarted","Data":"dc96ffde5a75f48f3a08a4123b33e74e019f564b756ecc04cb88f88f963835ab"} Jul 06 23:49:06.382246 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.382227 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gjb9z" event={"ID":"dc7855c6-e0ea-4385-b08f-a91bfb641524","Type":"ContainerStarted","Data":"0666820635dfdc21eca1e05613989d423e93dc08643cd7444cc5ca0cbb799af0"} Jul 06 23:49:06.392224 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.392164 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/243e7b42-f551-47ad-a169-c0fc8df2ca2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-4p46b\" (UID: \"243e7b42-f551-47ad-a169-c0fc8df2ca2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" Jul 06 23:49:06.394368 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.394345 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/243e7b42-f551-47ad-a169-c0fc8df2ca2d-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-4p46b\" (UID: \"243e7b42-f551-47ad-a169-c0fc8df2ca2d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" Jul 06 23:49:06.401365 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.401319 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" podStartSLOduration=2.401301765 podStartE2EDuration="2.401301765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:06.400334293 +0000 UTC m=+56.663006489" watchObservedRunningTime="2026-07-06 23:49:06.401301765 +0000 UTC m=+56.663973961" Jul 06 23:49:06.500178 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.500146 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" Jul 06 23:49:06.657590 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.656412 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b"] Jul 06 23:49:06.664044 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:06.664014 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243e7b42_f551_47ad_a169_c0fc8df2ca2d.slice/crio-76688f19210e50ae2fddef11d1107216fffe621802799e9169284218ebda4b38 WatchSource:0}: Error finding container 76688f19210e50ae2fddef11d1107216fffe621802799e9169284218ebda4b38: Status 404 returned error can't find the container with id 76688f19210e50ae2fddef11d1107216fffe621802799e9169284218ebda4b38 Jul 06 23:49:06.683455 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.682428 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-42rh9"] Jul 06 23:49:06.685365 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.684630 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-42rh9" Jul 06 23:49:06.688894 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.688567 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 23:49:06.688894 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.688629 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 23:49:06.688894 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.688573 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-q2t7l\"" Jul 06 23:49:06.696194 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.696078 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-42rh9"] Jul 06 23:49:06.797030 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.796433 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwxqf\" (UniqueName: \"kubernetes.io/projected/05df0493-f960-4a6a-b620-e532c61e18de-kube-api-access-mwxqf\") pod \"downloads-685bfb5f64-42rh9\" (UID: \"05df0493-f960-4a6a-b620-e532c61e18de\") " pod="openshift-console/downloads-685bfb5f64-42rh9" Jul 06 23:49:06.898035 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.897622 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwxqf\" (UniqueName: \"kubernetes.io/projected/05df0493-f960-4a6a-b620-e532c61e18de-kube-api-access-mwxqf\") pod \"downloads-685bfb5f64-42rh9\" (UID: \"05df0493-f960-4a6a-b620-e532c61e18de\") " pod="openshift-console/downloads-685bfb5f64-42rh9" Jul 06 23:49:06.909234 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.909163 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwxqf\" (UniqueName: \"kubernetes.io/projected/05df0493-f960-4a6a-b620-e532c61e18de-kube-api-access-mwxqf\") pod \"downloads-685bfb5f64-42rh9\" (UID: \"05df0493-f960-4a6a-b620-e532c61e18de\") " pod="openshift-console/downloads-685bfb5f64-42rh9" Jul 06 23:49:06.998904 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:06.998655 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-42rh9" Jul 06 23:49:07.176234 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.175266 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:49:07.176234 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.175749 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:49:07.177867 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.177782 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:49:07.178376 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.178349 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kjdfk\"" Jul 06 23:49:07.179111 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.179092 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m28zf\"" Jul 06 23:49:07.179420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.179404 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:49:07.179681 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.179665 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:49:07.189820 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.189797 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-42rh9"] Jul 06 23:49:07.194804 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:07.194780 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05df0493_f960_4a6a_b620_e532c61e18de.slice/crio-d590affe3ab0fe3fac1c31ebefec6cd44811b7152b4955bfa718f03b1ed001c4 WatchSource:0}: Error finding container d590affe3ab0fe3fac1c31ebefec6cd44811b7152b4955bfa718f03b1ed001c4: Status 404 returned error can't find the container with id d590affe3ab0fe3fac1c31ebefec6cd44811b7152b4955bfa718f03b1ed001c4 Jul 06 23:49:07.387017 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.386959 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-42rh9" event={"ID":"05df0493-f960-4a6a-b620-e532c61e18de","Type":"ContainerStarted","Data":"d590affe3ab0fe3fac1c31ebefec6cd44811b7152b4955bfa718f03b1ed001c4"} Jul 06 23:49:07.389149 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:07.389076 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" event={"ID":"243e7b42-f551-47ad-a169-c0fc8df2ca2d","Type":"ContainerStarted","Data":"76688f19210e50ae2fddef11d1107216fffe621802799e9169284218ebda4b38"} Jul 06 23:49:11.325101 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:11.325056 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j7xl5" Jul 06 23:49:13.410770 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.410678 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5pg5w" event={"ID":"0812878f-d9b5-4456-8b62-9a020d4cc2a4","Type":"ContainerStarted","Data":"66f6ac264bc19f62067906f8ad10b634582fa6d8aa956995f907fd6ca4775178"} Jul 06 23:49:13.412444 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.412417 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" event={"ID":"243e7b42-f551-47ad-a169-c0fc8df2ca2d","Type":"ContainerStarted","Data":"65e7482c6658a9113f2bc683d38cb8f229950d305320506f86272e16ca1c0657"} Jul 06 23:49:13.412621 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.412598 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" Jul 06 23:49:13.414662 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.414627 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gjb9z" event={"ID":"dc7855c6-e0ea-4385-b08f-a91bfb641524","Type":"ContainerStarted","Data":"5c43a9faa49a3a3411cb563286fac65edee040a9038f01ee2b0823b8d6c12d56"} Jul 06 23:49:13.414792 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.414680 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gjb9z" event={"ID":"dc7855c6-e0ea-4385-b08f-a91bfb641524","Type":"ContainerStarted","Data":"d70470f757605626b313674175c1e3b79daa57b8c15a5a8d7280cf2dc6fac76d"} Jul 06 23:49:13.416774 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.416621 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bs4cn" event={"ID":"41c2146f-78d2-499c-81f3-ae76240f8c62","Type":"ContainerStarted","Data":"151cd938de0cc88292929c11f908be7f80d1f2debb87c52816d0bf70473776ec"} Jul 06 23:49:13.416892 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.416795 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bs4cn" event={"ID":"41c2146f-78d2-499c-81f3-ae76240f8c62","Type":"ContainerStarted","Data":"5842e865e5677be935001224859df784b6e46e79afc70895462f1262cdf6ac20"} Jul 06 23:49:13.416973 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.416927 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:13.418241 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.418217 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" Jul 06 23:49:13.427290 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.427243 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5pg5w" podStartSLOduration=2.235528547 podStartE2EDuration="8.427229253s" totalImagesPullingTime="6.191700706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:13.426389749 +0000 UTC m=+63.689061944" watchObservedRunningTime="2026-07-06 23:49:13.427229253 +0000 UTC m=+63.689901448" Jul 06 23:49:13.441858 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.441799 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-4p46b" podStartSLOduration=1.793086398 podStartE2EDuration="7.441781257s" totalImagesPullingTime="5.648694859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:13.440170845 +0000 UTC m=+63.702843044" watchObservedRunningTime="2026-07-06 23:49:13.441781257 +0000 UTC m=+63.704453452" Jul 06 23:49:13.456721 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:13.456675 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bs4cn" podStartSLOduration=2.332484376 podStartE2EDuration="8.456656327s" totalImagesPullingTime="6.124171951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:13.455500591 +0000 UTC m=+63.718172786" watchObservedRunningTime="2026-07-06 23:49:13.456656327 +0000 UTC m=+63.719328521" Jul 06 23:49:15.975048 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.975009 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:49:15.975048 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.975051 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:49:15.975615 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.975114 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:49:15.977703 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.977679 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 23:49:15.977985 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.977961 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 23:49:15.978096 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.977961 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 23:49:15.987788 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.987635 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 23:49:15.988003 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.987981 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d9114c9-cf23-440f-a966-794fe592ced5-original-pull-secret\") pod \"global-pull-secret-syncer-vcbhn\" (UID: \"9d9114c9-cf23-440f-a966-794fe592ced5\") " pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:49:15.988143 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.988119 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5e0b8500-f7ec-40ad-885d-e358f1195532-metrics-certs\") pod \"network-metrics-daemon-nhbt9\" (UID: \"5e0b8500-f7ec-40ad-885d-e358f1195532\") " pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:49:15.999001 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:15.998974 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxbgj\" (UniqueName: \"kubernetes.io/projected/5b6424b2-15c0-472d-8e25-4e69aab7538a-kube-api-access-hxbgj\") pod \"network-check-target-sfsws\" (UID: \"5b6424b2-15c0-472d-8e25-4e69aab7538a\") " pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:49:16.091329 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.091294 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vcbhn" Jul 06 23:49:16.194739 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.194708 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-m28zf\"" Jul 06 23:49:16.202737 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.202714 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:49:16.211197 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.211038 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kjdfk\"" Jul 06 23:49:16.219002 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.218977 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nhbt9" Jul 06 23:49:16.368425 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.368356 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sfsws"] Jul 06 23:49:16.375092 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:16.375037 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b6424b2_15c0_472d_8e25_4e69aab7538a.slice/crio-ce9c00fb22e5cda29be3116f9032b554739ad396d1b6849547a4da2d8b2b9a84 WatchSource:0}: Error finding container ce9c00fb22e5cda29be3116f9032b554739ad396d1b6849547a4da2d8b2b9a84: Status 404 returned error can't find the container with id ce9c00fb22e5cda29be3116f9032b554739ad396d1b6849547a4da2d8b2b9a84 Jul 06 23:49:16.391123 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.391054 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vcbhn"] Jul 06 23:49:16.393674 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:16.393646 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9114c9_cf23_440f_a966_794fe592ced5.slice/crio-30e5609e5cf511702bae61c26d6b0d4c507ff01919c66fe828cf133d317b80c9 WatchSource:0}: Error finding container 30e5609e5cf511702bae61c26d6b0d4c507ff01919c66fe828cf133d317b80c9: Status 404 returned error can't find the container with id 30e5609e5cf511702bae61c26d6b0d4c507ff01919c66fe828cf133d317b80c9 Jul 06 23:49:16.415990 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.415956 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nhbt9"] Jul 06 23:49:16.419109 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:16.419085 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0b8500_f7ec_40ad_885d_e358f1195532.slice/crio-fae866ce356a96eb29e73f56efaa231b05715e25d16777e2a9cdbe2d9c46ae59 WatchSource:0}: Error finding container fae866ce356a96eb29e73f56efaa231b05715e25d16777e2a9cdbe2d9c46ae59: Status 404 returned error can't find the container with id fae866ce356a96eb29e73f56efaa231b05715e25d16777e2a9cdbe2d9c46ae59 Jul 06 23:49:16.428731 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.428674 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sfsws" event={"ID":"5b6424b2-15c0-472d-8e25-4e69aab7538a","Type":"ContainerStarted","Data":"ce9c00fb22e5cda29be3116f9032b554739ad396d1b6849547a4da2d8b2b9a84"} Jul 06 23:49:16.429907 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.429885 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nhbt9" event={"ID":"5e0b8500-f7ec-40ad-885d-e358f1195532","Type":"ContainerStarted","Data":"fae866ce356a96eb29e73f56efaa231b05715e25d16777e2a9cdbe2d9c46ae59"} Jul 06 23:49:16.432240 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.432219 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gjb9z" event={"ID":"dc7855c6-e0ea-4385-b08f-a91bfb641524","Type":"ContainerStarted","Data":"37868b522b008cac6a394c67ca68cc2f4fd0322f53c0868a9a1f841f5b18c4ee"} Jul 06 23:49:16.433520 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.433492 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcbhn" event={"ID":"9d9114c9-cf23-440f-a966-794fe592ced5","Type":"ContainerStarted","Data":"30e5609e5cf511702bae61c26d6b0d4c507ff01919c66fe828cf133d317b80c9"} Jul 06 23:49:16.450062 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:16.450015 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gjb9z" podStartSLOduration=1.519847973 podStartE2EDuration="11.449999063s" totalImagesPullingTime="9.93015109s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:49:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:16.449282019 +0000 UTC m=+66.711954214" watchObservedRunningTime="2026-07-06 23:49:16.449999063 +0000 UTC m=+66.712671249" Jul 06 23:49:17.185151 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.185113 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c8c7f86c4-d6qw6"] Jul 06 23:49:17.205281 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.205251 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.206832 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.206375 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8c7f86c4-d6qw6"] Jul 06 23:49:17.214389 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.208499 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 23:49:17.214389 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.208742 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 23:49:17.214389 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.209380 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s7hfr\"" Jul 06 23:49:17.214389 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.209628 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 23:49:17.214389 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.209835 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 23:49:17.214389 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.210049 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 23:49:17.215519 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.215169 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 23:49:17.285761 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.285716 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm8mb\" (UniqueName: \"kubernetes.io/projected/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-kube-api-access-jm8mb\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.285761 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.285768 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-oauth-serving-cert\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.286012 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.285805 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-service-ca\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.286012 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.285843 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-trusted-ca-bundle\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.286012 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.285869 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-serving-cert\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.286012 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.285895 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-oauth-config\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.286012 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.285920 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-config\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.386496 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.386443 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm8mb\" (UniqueName: \"kubernetes.io/projected/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-kube-api-access-jm8mb\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.386668 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.386512 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-oauth-serving-cert\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.386668 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.386546 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-service-ca\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.386668 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.386581 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-trusted-ca-bundle\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.386668 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.386607 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-serving-cert\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.386668 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.386634 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-oauth-config\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.386668 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.386661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-config\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.388922 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.387534 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-config\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.388922 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.388400 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-oauth-serving-cert\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.389883 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.389700 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-service-ca\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.389883 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.389700 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-trusted-ca-bundle\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.395133 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.395088 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-serving-cert\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.398846 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.398209 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-oauth-config\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.398846 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.398809 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm8mb\" (UniqueName: \"kubernetes.io/projected/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-kube-api-access-jm8mb\") pod \"console-c8c7f86c4-d6qw6\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:17.526206 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:17.525878 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:18.036464 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:18.036361 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8c7f86c4-d6qw6"] Jul 06 23:49:18.044584 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:18.043995 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5953c9_4e87_46e8_b6a7_f9aa78ae893d.slice/crio-44b9663d50d74e9b39f40abd0f86de8651f89dd0762decf7cf71a4d5ed46e075 WatchSource:0}: Error finding container 44b9663d50d74e9b39f40abd0f86de8651f89dd0762decf7cf71a4d5ed46e075: Status 404 returned error can't find the container with id 44b9663d50d74e9b39f40abd0f86de8651f89dd0762decf7cf71a4d5ed46e075 Jul 06 23:49:18.446120 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:18.445960 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nhbt9" event={"ID":"5e0b8500-f7ec-40ad-885d-e358f1195532","Type":"ContainerStarted","Data":"be88d5b934533a6a4442bfd4bbae4d91aeb7f449768497fb63c3943f4406ebd0"} Jul 06 23:49:18.446120 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:18.446025 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nhbt9" event={"ID":"5e0b8500-f7ec-40ad-885d-e358f1195532","Type":"ContainerStarted","Data":"567a0b6da10bde06b183ccf2251572eb0be2ca51e3702f5568e2ecaa9e47f30b"} Jul 06 23:49:18.447799 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:18.447760 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8c7f86c4-d6qw6" event={"ID":"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d","Type":"ContainerStarted","Data":"44b9663d50d74e9b39f40abd0f86de8651f89dd0762decf7cf71a4d5ed46e075"} Jul 06 23:49:18.464113 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:18.463612 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nhbt9" podStartSLOduration=66.982831055 podStartE2EDuration="1m8.463592334s" totalImagesPullingTime="1.480761279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:18.463119874 +0000 UTC m=+68.725792071" watchObservedRunningTime="2026-07-06 23:49:18.463592334 +0000 UTC m=+68.726264530" Jul 06 23:49:21.568623 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.568582 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs"] Jul 06 23:49:21.570837 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.570806 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.573991 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.573821 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-8p8z5\"" Jul 06 23:49:21.573991 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.573895 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 23:49:21.574521 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.574501 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 23:49:21.574632 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.574529 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 23:49:21.574632 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.574547 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 23:49:21.574731 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.574646 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 23:49:21.585393 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.585355 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs"] Jul 06 23:49:21.612780 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.612748 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5l7sg"] Jul 06 23:49:21.615028 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.615002 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.618355 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.618167 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 23:49:21.618355 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.618314 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 23:49:21.618626 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.618390 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q9bcb\"" Jul 06 23:49:21.618626 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.618620 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 23:49:21.718371 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718341 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-textfile\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.718371 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718374 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fefc3ede-6c89-468a-80e4-e03f1f9fc913-metrics-client-ca\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.718645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718396 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.718645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718428 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fefc3ede-6c89-468a-80e4-e03f1f9fc913-sys\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.718645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718452 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.718645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718491 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64cg9\" (UniqueName: \"kubernetes.io/projected/fefc3ede-6c89-468a-80e4-e03f1f9fc913-kube-api-access-64cg9\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.718645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718515 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.718645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718538 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jg4v\" (UniqueName: \"kubernetes.io/projected/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-kube-api-access-6jg4v\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.718645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718570 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fefc3ede-6c89-468a-80e4-e03f1f9fc913-root\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.718645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718589 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-wtmp\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.718645 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718611 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-accelerators-collector-config\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.718998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718705 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-tls\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.718998 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.718750 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.819283 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819201 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-wtmp\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819283 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819241 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-accelerators-collector-config\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819364 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-tls\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819399 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.819534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819400 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-wtmp\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819442 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-textfile\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819466 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fefc3ede-6c89-468a-80e4-e03f1f9fc913-metrics-client-ca\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819518 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.819852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819551 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fefc3ede-6c89-468a-80e4-e03f1f9fc913-sys\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819588 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819616 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64cg9\" (UniqueName: \"kubernetes.io/projected/fefc3ede-6c89-468a-80e4-e03f1f9fc913-kube-api-access-64cg9\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.819852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819674 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jg4v\" (UniqueName: \"kubernetes.io/projected/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-kube-api-access-6jg4v\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.819852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819703 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fefc3ede-6c89-468a-80e4-e03f1f9fc913-root\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819766 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-textfile\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.819852 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819781 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fefc3ede-6c89-468a-80e4-e03f1f9fc913-root\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.820207 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819910 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-accelerators-collector-config\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.820207 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.819959 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fefc3ede-6c89-468a-80e4-e03f1f9fc913-sys\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.820207 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.820048 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fefc3ede-6c89-468a-80e4-e03f1f9fc913-metrics-client-ca\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.820617 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.820573 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.822354 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.822329 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.822465 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.822352 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.822465 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.822436 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fefc3ede-6c89-468a-80e4-e03f1f9fc913-node-exporter-tls\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.822827 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.822805 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.828250 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.828227 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64cg9\" (UniqueName: \"kubernetes.io/projected/fefc3ede-6c89-468a-80e4-e03f1f9fc913-kube-api-access-64cg9\") pod \"node-exporter-5l7sg\" (UID: \"fefc3ede-6c89-468a-80e4-e03f1f9fc913\") " pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:21.828588 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.828569 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jg4v\" (UniqueName: \"kubernetes.io/projected/5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f-kube-api-access-6jg4v\") pod \"openshift-state-metrics-54cf96dfcf-bxlbs\" (UID: \"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.881514 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.881467 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" Jul 06 23:49:21.926318 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:21.926287 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5l7sg" Jul 06 23:49:23.423803 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:23.423771 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bs4cn" Jul 06 23:49:24.572775 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.572741 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5f56c4868-62rgg"] Jul 06 23:49:24.576990 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.576969 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.579598 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.579565 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 23:49:24.579598 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.579591 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 23:49:24.579786 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.579731 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6xvjk\"" Jul 06 23:49:24.579845 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.579740 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 23:49:24.579895 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.579857 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 23:49:24.580147 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.580122 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 23:49:24.580431 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.580411 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2edfgihiqdg01\"" Jul 06 23:49:24.594999 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.594972 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f56c4868-62rgg"] Jul 06 23:49:24.645727 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.645687 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.645727 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.645728 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.645969 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.645753 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-grpc-tls\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.645969 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.645814 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-tls\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.645969 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.645852 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.645969 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.645895 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9n7k\" (UniqueName: \"kubernetes.io/projected/94654697-ef15-4f75-b0b4-3a3f9023e3a0-kube-api-access-c9n7k\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.646146 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.645975 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.646146 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.646008 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94654697-ef15-4f75-b0b4-3a3f9023e3a0-metrics-client-ca\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.746731 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.746690 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.746927 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.746740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94654697-ef15-4f75-b0b4-3a3f9023e3a0-metrics-client-ca\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.746927 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.746787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.746927 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.746825 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.746927 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.746854 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-grpc-tls\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.746927 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.746888 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-tls\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.746927 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.746918 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.747216 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.746955 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9n7k\" (UniqueName: \"kubernetes.io/projected/94654697-ef15-4f75-b0b4-3a3f9023e3a0-kube-api-access-c9n7k\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.747761 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.747623 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94654697-ef15-4f75-b0b4-3a3f9023e3a0-metrics-client-ca\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.749861 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.749840 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.750222 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.750204 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.750432 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.750381 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.750564 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.750538 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.750638 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.750545 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-grpc-tls\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.750839 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.750815 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/94654697-ef15-4f75-b0b4-3a3f9023e3a0-secret-thanos-querier-tls\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.757289 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.757271 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9n7k\" (UniqueName: \"kubernetes.io/projected/94654697-ef15-4f75-b0b4-3a3f9023e3a0-kube-api-access-c9n7k\") pod \"thanos-querier-5f56c4868-62rgg\" (UID: \"94654697-ef15-4f75-b0b4-3a3f9023e3a0\") " pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:24.888718 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:24.888628 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:26.350103 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.350063 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n"] Jul 06 23:49:26.419345 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.419315 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n"] Jul 06 23:49:26.419526 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.419451 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:26.421801 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.421618 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 23:49:26.421900 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.421807 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-brwtm\"" Jul 06 23:49:26.562706 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.562670 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c92b6d08-724e-41aa-bafb-4f0b9fbe5628-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ghb2n\" (UID: \"c92b6d08-724e-41aa-bafb-4f0b9fbe5628\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:26.562907 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.562728 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdkgt\" (UniqueName: \"kubernetes.io/projected/c92b6d08-724e-41aa-bafb-4f0b9fbe5628-kube-api-access-zdkgt\") pod \"monitoring-plugin-d5dcff7cf-ghb2n\" (UID: \"c92b6d08-724e-41aa-bafb-4f0b9fbe5628\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:26.664856 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.663921 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c92b6d08-724e-41aa-bafb-4f0b9fbe5628-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ghb2n\" (UID: \"c92b6d08-724e-41aa-bafb-4f0b9fbe5628\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:26.664856 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.663978 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdkgt\" (UniqueName: \"kubernetes.io/projected/c92b6d08-724e-41aa-bafb-4f0b9fbe5628-kube-api-access-zdkgt\") pod \"monitoring-plugin-d5dcff7cf-ghb2n\" (UID: \"c92b6d08-724e-41aa-bafb-4f0b9fbe5628\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:26.672179 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.672146 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c92b6d08-724e-41aa-bafb-4f0b9fbe5628-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-ghb2n\" (UID: \"c92b6d08-724e-41aa-bafb-4f0b9fbe5628\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:26.673369 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.673329 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdkgt\" (UniqueName: \"kubernetes.io/projected/c92b6d08-724e-41aa-bafb-4f0b9fbe5628-kube-api-access-zdkgt\") pod \"monitoring-plugin-d5dcff7cf-ghb2n\" (UID: \"c92b6d08-724e-41aa-bafb-4f0b9fbe5628\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:26.732323 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.732289 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:26.819498 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.819445 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh"] Jul 06 23:49:26.829742 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.829717 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:26.832313 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.832261 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 23:49:26.832313 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.832265 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 23:49:26.833311 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.832581 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 23:49:26.833311 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.832837 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 23:49:26.833311 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.832920 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-rbkx8\"" Jul 06 23:49:26.833311 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.833146 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 23:49:26.837587 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.837565 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 23:49:26.840603 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.840558 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh"] Jul 06 23:49:26.967337 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.967252 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9rfj\" (UniqueName: \"kubernetes.io/projected/19879c39-6c1e-4cdf-b480-a3c004613a96-kube-api-access-l9rfj\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:26.967337 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.967302 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19879c39-6c1e-4cdf-b480-a3c004613a96-serving-certs-ca-bundle\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:26.967585 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.967343 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-federate-client-tls\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:26.967585 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.967379 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19879c39-6c1e-4cdf-b480-a3c004613a96-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:26.967585 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.967413 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-telemeter-client-tls\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:26.967585 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.967452 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19879c39-6c1e-4cdf-b480-a3c004613a96-metrics-client-ca\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:26.967585 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.967501 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-secret-telemeter-client\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:26.967585 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:26.967549 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.068336 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.068294 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19879c39-6c1e-4cdf-b480-a3c004613a96-metrics-client-ca\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.068568 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.068343 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-secret-telemeter-client\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.068568 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.068382 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.068568 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.068417 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9rfj\" (UniqueName: \"kubernetes.io/projected/19879c39-6c1e-4cdf-b480-a3c004613a96-kube-api-access-l9rfj\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.068568 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.068444 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19879c39-6c1e-4cdf-b480-a3c004613a96-serving-certs-ca-bundle\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.068797 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.068592 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-federate-client-tls\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.068797 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.068637 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19879c39-6c1e-4cdf-b480-a3c004613a96-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.068797 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.068670 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-telemeter-client-tls\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.069134 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.069088 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19879c39-6c1e-4cdf-b480-a3c004613a96-metrics-client-ca\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.069134 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.069117 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19879c39-6c1e-4cdf-b480-a3c004613a96-serving-certs-ca-bundle\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.069687 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.069664 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19879c39-6c1e-4cdf-b480-a3c004613a96-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.071073 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.071049 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-telemeter-client-tls\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.071168 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.071097 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.071307 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.071285 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-federate-client-tls\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.071347 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.071294 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/19879c39-6c1e-4cdf-b480-a3c004613a96-secret-telemeter-client\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.079377 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.079354 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9rfj\" (UniqueName: \"kubernetes.io/projected/19879c39-6c1e-4cdf-b480-a3c004613a96-kube-api-access-l9rfj\") pod \"telemeter-client-7cfd57dc85-qlqqh\" (UID: \"19879c39-6c1e-4cdf-b480-a3c004613a96\") " pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.142567 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.142533 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" Jul 06 23:49:27.395402 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:27.395378 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-755c4fcb8f-pwv98" Jul 06 23:49:29.676900 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:29.676854 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfefc3ede_6c89_468a_80e4_e03f1f9fc913.slice/crio-cd74f605d41c9f0f8ab96bff0e862b5f832b669a9e772c50c510694b727173e7 WatchSource:0}: Error finding container cd74f605d41c9f0f8ab96bff0e862b5f832b669a9e772c50c510694b727173e7: Status 404 returned error can't find the container with id cd74f605d41c9f0f8ab96bff0e862b5f832b669a9e772c50c510694b727173e7 Jul 06 23:49:29.893704 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:29.893609 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5f56c4868-62rgg"] Jul 06 23:49:29.899158 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:29.897635 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n"] Jul 06 23:49:29.929409 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:29.929247 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh"] Jul 06 23:49:29.935345 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:29.935312 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19879c39_6c1e_4cdf_b480_a3c004613a96.slice/crio-4595977c6e191a76194e2ecd24db196e6693c0e9f7a5c13e8317cef0fb8bf63d WatchSource:0}: Error finding container 4595977c6e191a76194e2ecd24db196e6693c0e9f7a5c13e8317cef0fb8bf63d: Status 404 returned error can't find the container with id 4595977c6e191a76194e2ecd24db196e6693c0e9f7a5c13e8317cef0fb8bf63d Jul 06 23:49:29.943674 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:29.943650 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs"] Jul 06 23:49:29.947699 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:49:29.947678 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdd4fde_44da_40d4_b4a4_5ff1f25f2a0f.slice/crio-bbabb78b3cdadef9464870adb07e689f1d6998dbb531122a3f5272d2a69cacdb WatchSource:0}: Error finding container bbabb78b3cdadef9464870adb07e689f1d6998dbb531122a3f5272d2a69cacdb: Status 404 returned error can't find the container with id bbabb78b3cdadef9464870adb07e689f1d6998dbb531122a3f5272d2a69cacdb Jul 06 23:49:30.506312 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.505578 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vcbhn" event={"ID":"9d9114c9-cf23-440f-a966-794fe592ced5","Type":"ContainerStarted","Data":"56fcc8829754e80944d8e032c5c65ad99ca6d9a3b0db2b21371dbf7526e3fdc3"} Jul 06 23:49:30.510911 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.510748 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" event={"ID":"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f","Type":"ContainerStarted","Data":"b2d99403fc498c3413167c73b7497e689da549943db778887902222857d3d1f1"} Jul 06 23:49:30.510911 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.510812 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" event={"ID":"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f","Type":"ContainerStarted","Data":"97a685f29e82f38259dc9daea1b0cf5e4d2831f3c9e5dd1ad76aacb76d742696"} Jul 06 23:49:30.510911 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.510840 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" event={"ID":"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f","Type":"ContainerStarted","Data":"bbabb78b3cdadef9464870adb07e689f1d6998dbb531122a3f5272d2a69cacdb"} Jul 06 23:49:30.515209 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.515174 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8c7f86c4-d6qw6" event={"ID":"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d","Type":"ContainerStarted","Data":"d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c"} Jul 06 23:49:30.517537 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.517419 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" event={"ID":"19879c39-6c1e-4cdf-b480-a3c004613a96","Type":"ContainerStarted","Data":"4595977c6e191a76194e2ecd24db196e6693c0e9f7a5c13e8317cef0fb8bf63d"} Jul 06 23:49:30.521770 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.521719 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sfsws" event={"ID":"5b6424b2-15c0-472d-8e25-4e69aab7538a","Type":"ContainerStarted","Data":"af84ac853740f2a6a06926efc08b6cafab791293e7b04c1b74f9341c0f492a9f"} Jul 06 23:49:30.521955 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.521802 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:49:30.523326 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.523293 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" event={"ID":"c92b6d08-724e-41aa-bafb-4f0b9fbe5628","Type":"ContainerStarted","Data":"2c12d856e5416a58611a0125f9af52dc5ff32139cfb86fc5bed694d152c644b5"} Jul 06 23:49:30.525726 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.525583 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" event={"ID":"94654697-ef15-4f75-b0b4-3a3f9023e3a0","Type":"ContainerStarted","Data":"6a418c256cd26b1b5505eb5f6368a19dcd6650636d2032c2fc7c131c7408dc0b"} Jul 06 23:49:30.527725 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.527664 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5l7sg" event={"ID":"fefc3ede-6c89-468a-80e4-e03f1f9fc913","Type":"ContainerStarted","Data":"cd74f605d41c9f0f8ab96bff0e862b5f832b669a9e772c50c510694b727173e7"} Jul 06 23:49:30.529781 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.529758 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-42rh9" event={"ID":"05df0493-f960-4a6a-b620-e532c61e18de","Type":"ContainerStarted","Data":"237ee02e191778a9943e53d5ad847168e89430d933d9dec5d1a68c52b040f696"} Jul 06 23:49:30.530424 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.530131 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-42rh9" Jul 06 23:49:30.537081 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.537009 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vcbhn" podStartSLOduration=66.260833602 podStartE2EDuration="1m19.536992103s" totalImagesPullingTime="13.276158501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:30.52140836 +0000 UTC m=+80.784080557" watchObservedRunningTime="2026-07-06 23:49:30.536992103 +0000 UTC m=+80.799664296" Jul 06 23:49:30.537429 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.537394 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c8c7f86c4-d6qw6" podStartSLOduration=1.872062471 podStartE2EDuration="13.537382379s" totalImagesPullingTime="11.665319908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:30.536909193 +0000 UTC m=+80.799581391" watchObservedRunningTime="2026-07-06 23:49:30.537382379 +0000 UTC m=+80.800054575" Jul 06 23:49:30.542870 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.542820 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-42rh9" Jul 06 23:49:30.551885 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.551837 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-42rh9" podStartSLOduration=2.082651827 podStartE2EDuration="24.551816994s" totalImagesPullingTime="22.469165167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:30.550632502 +0000 UTC m=+80.813304707" watchObservedRunningTime="2026-07-06 23:49:30.551816994 +0000 UTC m=+80.814489190" Jul 06 23:49:30.567313 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:30.567254 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sfsws" podStartSLOduration=67.272729474 podStartE2EDuration="1m20.567234236s" totalImagesPullingTime="13.294504762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:48:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:30.566430149 +0000 UTC m=+80.829102344" watchObservedRunningTime="2026-07-06 23:49:30.567234236 +0000 UTC m=+80.829906430" Jul 06 23:49:31.541337 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:31.541260 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5l7sg" event={"ID":"fefc3ede-6c89-468a-80e4-e03f1f9fc913","Type":"ContainerStarted","Data":"83346d9d7c8f93c15296aeba6c801af796c23140dcbceb1aaf26e91a814e3614"} Jul 06 23:49:32.549851 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:32.549811 2583 generic.go:356] "Generic (PLEG): container finished" podID="fefc3ede-6c89-468a-80e4-e03f1f9fc913" containerID="83346d9d7c8f93c15296aeba6c801af796c23140dcbceb1aaf26e91a814e3614" exitCode=0 Jul 06 23:49:32.550893 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:32.550817 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5l7sg" event={"ID":"fefc3ede-6c89-468a-80e4-e03f1f9fc913","Type":"ContainerDied","Data":"83346d9d7c8f93c15296aeba6c801af796c23140dcbceb1aaf26e91a814e3614"} Jul 06 23:49:35.564127 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.564078 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5l7sg" event={"ID":"fefc3ede-6c89-468a-80e4-e03f1f9fc913","Type":"ContainerStarted","Data":"e341cee8beb976601c9b9df2e65d0bfbcf31f3e48c8085205f088f9650a8870f"} Jul 06 23:49:35.564771 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.564154 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5l7sg" event={"ID":"fefc3ede-6c89-468a-80e4-e03f1f9fc913","Type":"ContainerStarted","Data":"67c10d8e747c7962b14f5fa69a10dca40288b5841d16a6b4f9a2b47c4873e505"} Jul 06 23:49:35.566704 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.566611 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" event={"ID":"5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f","Type":"ContainerStarted","Data":"45f2890df675f381f5c1df67ecdc2ff2f86df014523d656b2540a84e772a2243"} Jul 06 23:49:35.568428 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.568403 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" event={"ID":"19879c39-6c1e-4cdf-b480-a3c004613a96","Type":"ContainerStarted","Data":"80c93d82eb04f202bb6ecba0176bf2019bbefaa4afe07d8b47ab893df1d109e6"} Jul 06 23:49:35.570087 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.570058 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" event={"ID":"c92b6d08-724e-41aa-bafb-4f0b9fbe5628","Type":"ContainerStarted","Data":"6bf25a268acde28b61baa09c89bb64b988d193be75504c1806cab26ac6160567"} Jul 06 23:49:35.570212 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.570190 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:35.572841 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.572812 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" event={"ID":"94654697-ef15-4f75-b0b4-3a3f9023e3a0","Type":"ContainerStarted","Data":"78d0855b840cb3ae428845897b83bc1714e1536e8b08da4189c7079963b7bd5f"} Jul 06 23:49:35.572934 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.572871 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" event={"ID":"94654697-ef15-4f75-b0b4-3a3f9023e3a0","Type":"ContainerStarted","Data":"89eceb1399c57f8ae08dbfd9fd25986c2a19280663f1d10a7268bdac8a316406"} Jul 06 23:49:35.572934 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.572899 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" event={"ID":"94654697-ef15-4f75-b0b4-3a3f9023e3a0","Type":"ContainerStarted","Data":"f9281f90707f438eea5e7a7113c2bd3458b0e39ba9b3335374e5bc9ef4e0a04f"} Jul 06 23:49:35.576008 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.575988 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" Jul 06 23:49:35.589719 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.589669 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5l7sg" podStartSLOduration=12.90706958 podStartE2EDuration="14.589650623s" totalImagesPullingTime="1.515005893s" totalInitContainerRuntime="167.57515ms" isStatefulPod=true podCreationTimestamp="2026-07-06 23:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:35.588330463 +0000 UTC m=+85.851002661" watchObservedRunningTime="2026-07-06 23:49:35.589650623 +0000 UTC m=+85.852322797" Jul 06 23:49:35.603770 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.603731 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-ghb2n" podStartSLOduration=4.561716651 podStartE2EDuration="9.603711503s" totalImagesPullingTime="5.041994852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:35.602525943 +0000 UTC m=+85.865198139" watchObservedRunningTime="2026-07-06 23:49:35.603711503 +0000 UTC m=+85.866383695" Jul 06 23:49:35.624338 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:35.624285 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-bxlbs" podStartSLOduration=9.762546201 podStartE2EDuration="14.624267304s" totalImagesPullingTime="4.861721103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:35.622650667 +0000 UTC m=+85.885322874" watchObservedRunningTime="2026-07-06 23:49:35.624267304 +0000 UTC m=+85.886939498" Jul 06 23:49:36.864263 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:36.864222 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c8c7f86c4-d6qw6"] Jul 06 23:49:37.526876 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:37.526832 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:49:37.585043 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:37.585003 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" event={"ID":"19879c39-6c1e-4cdf-b480-a3c004613a96","Type":"ContainerStarted","Data":"dfa4cd5f125d059be178d5e659ff9dc5544887da4651481e11f3efa28e4114fc"} Jul 06 23:49:37.585246 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:37.585080 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" event={"ID":"19879c39-6c1e-4cdf-b480-a3c004613a96","Type":"ContainerStarted","Data":"1814591881de62990bee640322236c454480546f89c697b7962c52e5acb98c12"} Jul 06 23:49:37.588846 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:37.588816 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" event={"ID":"94654697-ef15-4f75-b0b4-3a3f9023e3a0","Type":"ContainerStarted","Data":"a31ba63a9b4b2bac8c238d272ea3636b7cda1d3cba0daa2dda8ae45c3a720da3"} Jul 06 23:49:37.588976 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:37.588891 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" event={"ID":"94654697-ef15-4f75-b0b4-3a3f9023e3a0","Type":"ContainerStarted","Data":"bcdac4f229e6843295afe180e3cc9f4d4a830ef904eab44c541e259c764ef6b9"} Jul 06 23:49:37.588976 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:37.588933 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" event={"ID":"94654697-ef15-4f75-b0b4-3a3f9023e3a0","Type":"ContainerStarted","Data":"80e9a57f1d8b5e497ed0d46e8dfcc9813fda122ce99de195157355d8ffa19221"} Jul 06 23:49:37.614535 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:37.614461 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7cfd57dc85-qlqqh" podStartSLOduration=4.63774393 podStartE2EDuration="11.61443986s" totalImagesPullingTime="6.97669593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:37.61308529 +0000 UTC m=+87.875757497" watchObservedRunningTime="2026-07-06 23:49:37.61443986 +0000 UTC m=+87.877112059" Jul 06 23:49:37.649319 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:37.649218 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" podStartSLOduration=6.950096084 podStartE2EDuration="13.649196675s" totalImagesPullingTime="6.699100591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:49:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:49:37.646513086 +0000 UTC m=+87.909185282" watchObservedRunningTime="2026-07-06 23:49:37.649196675 +0000 UTC m=+87.911868872" Jul 06 23:49:38.592893 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:38.592861 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:49:44.602373 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:49:44.602344 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5f56c4868-62rgg" Jul 06 23:50:01.546308 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:01.546273 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sfsws" Jul 06 23:50:01.888426 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:01.888337 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-c8c7f86c4-d6qw6" podUID="5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" containerName="console" containerID="cri-o://d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c" gracePeriod=15 Jul 06 23:50:02.161761 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.161684 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:50:02.251373 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.251333 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-config\") pod \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " Jul 06 23:50:02.251593 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.251383 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-serving-cert\") pod \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " Jul 06 23:50:02.251593 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.251406 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-oauth-config\") pod \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " Jul 06 23:50:02.251593 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.251436 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-service-ca\") pod \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " Jul 06 23:50:02.251593 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.251494 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm8mb\" (UniqueName: \"kubernetes.io/projected/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-kube-api-access-jm8mb\") pod \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " Jul 06 23:50:02.251593 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.251515 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-oauth-serving-cert\") pod \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " Jul 06 23:50:02.251593 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.251543 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-trusted-ca-bundle\") pod \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\" (UID: \"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d\") " Jul 06 23:50:02.251951 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.251921 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-config" (OuterVolumeSpecName: "console-config") pod "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" (UID: "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:02.252044 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.252017 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-service-ca" (OuterVolumeSpecName: "service-ca") pod "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" (UID: "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:02.252097 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.252046 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" (UID: "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:02.252185 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.252162 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" (UID: "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:50:02.253985 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.253954 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" (UID: "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:50:02.253985 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.253972 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-kube-api-access-jm8mb" (OuterVolumeSpecName: "kube-api-access-jm8mb") pod "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" (UID: "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d"). InnerVolumeSpecName "kube-api-access-jm8mb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:50:02.254123 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.253987 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" (UID: "5f5953c9-4e87-46e8-b6a7-f9aa78ae893d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:50:02.353046 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.353009 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-service-ca\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:50:02.353046 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.353039 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm8mb\" (UniqueName: \"kubernetes.io/projected/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-kube-api-access-jm8mb\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:50:02.353046 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.353049 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-oauth-serving-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:50:02.353276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.353058 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-trusted-ca-bundle\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:50:02.353276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.353068 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-config\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:50:02.353276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.353076 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-serving-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:50:02.353276 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.353085 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d-console-oauth-config\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:50:02.686912 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.686881 2583 generic.go:356] "Generic (PLEG): container finished" podID="5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" containerID="d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c" exitCode=0 Jul 06 23:50:02.687380 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.686927 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8c7f86c4-d6qw6" Jul 06 23:50:02.687380 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.686962 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8c7f86c4-d6qw6" event={"ID":"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d","Type":"ContainerDied","Data":"d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c"} Jul 06 23:50:02.687380 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.687002 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8c7f86c4-d6qw6" event={"ID":"5f5953c9-4e87-46e8-b6a7-f9aa78ae893d","Type":"ContainerDied","Data":"44b9663d50d74e9b39f40abd0f86de8651f89dd0762decf7cf71a4d5ed46e075"} Jul 06 23:50:02.687380 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.687022 2583 scope.go:117] "RemoveContainer" containerID="d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c" Jul 06 23:50:02.698537 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.698516 2583 scope.go:117] "RemoveContainer" containerID="d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c" Jul 06 23:50:02.698792 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:50:02.698765 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c\": container with ID starting with d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c not found: ID does not exist" containerID="d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c" Jul 06 23:50:02.698882 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.698798 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c"} err="failed to get container status \"d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c\": rpc error: code = NotFound desc = could not find container \"d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c\": container with ID starting with d06617e179684715efe3b0edb5751f9b3ab417fdc6e4afeb7cf98b0f9a1aa72c not found: ID does not exist" Jul 06 23:50:02.730338 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.730305 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c8c7f86c4-d6qw6"] Jul 06 23:50:02.743118 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:02.743087 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c8c7f86c4-d6qw6"] Jul 06 23:50:04.179536 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:04.179503 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" path="/var/lib/kubelet/pods/5f5953c9-4e87-46e8-b6a7-f9aa78ae893d/volumes" Jul 06 23:50:42.526184 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.526137 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-654cf8646-mttr5"] Jul 06 23:50:42.526846 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.526414 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" containerName="console" Jul 06 23:50:42.526846 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.526425 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" containerName="console" Jul 06 23:50:42.526846 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.526490 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f5953c9-4e87-46e8-b6a7-f9aa78ae893d" containerName="console" Jul 06 23:50:42.528800 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.528779 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.530894 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.530862 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 23:50:42.531005 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.530922 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 23:50:42.534982 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.534961 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s7hfr\"" Jul 06 23:50:42.535438 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.535417 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 23:50:42.535558 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.535239 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 23:50:42.535558 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.535026 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 23:50:42.539292 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.539271 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 23:50:42.547969 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.547948 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654cf8646-mttr5"] Jul 06 23:50:42.664540 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.664505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-config\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.664706 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.664552 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-serving-cert\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.664706 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.664590 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-oauth-config\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.664706 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.664606 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-oauth-serving-cert\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.664706 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.664632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-service-ca\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.664706 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.664696 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-trusted-ca-bundle\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.664865 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.664726 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x75gq\" (UniqueName: \"kubernetes.io/projected/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-kube-api-access-x75gq\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.765264 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.765227 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-trusted-ca-bundle\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.765264 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.765263 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x75gq\" (UniqueName: \"kubernetes.io/projected/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-kube-api-access-x75gq\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.765531 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.765286 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-config\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.765531 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.765337 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-serving-cert\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.765531 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.765375 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-oauth-config\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.765531 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.765398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-oauth-serving-cert\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.765531 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.765444 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-service-ca\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.766078 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.766056 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-config\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.766186 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.766153 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-service-ca\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.766248 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.766237 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-oauth-serving-cert\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.766305 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.766262 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-trusted-ca-bundle\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.767883 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.767857 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-serving-cert\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.767883 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.767878 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-oauth-config\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.772784 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.772759 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x75gq\" (UniqueName: \"kubernetes.io/projected/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-kube-api-access-x75gq\") pod \"console-654cf8646-mttr5\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.840015 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.839932 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:42.970060 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:42.970026 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-654cf8646-mttr5"] Jul 06 23:50:42.972991 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:50:42.972963 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f41d6c9_6d43_442c_901f_a6c89a4cf9c8.slice/crio-70c33bb6b744701e8bafa9215ebf6ccc8c2c23aaf27cf7623ed2db9e8cefd177 WatchSource:0}: Error finding container 70c33bb6b744701e8bafa9215ebf6ccc8c2c23aaf27cf7623ed2db9e8cefd177: Status 404 returned error can't find the container with id 70c33bb6b744701e8bafa9215ebf6ccc8c2c23aaf27cf7623ed2db9e8cefd177 Jul 06 23:50:43.825195 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:43.825156 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654cf8646-mttr5" event={"ID":"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8","Type":"ContainerStarted","Data":"12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990"} Jul 06 23:50:43.825566 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:43.825218 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654cf8646-mttr5" event={"ID":"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8","Type":"ContainerStarted","Data":"70c33bb6b744701e8bafa9215ebf6ccc8c2c23aaf27cf7623ed2db9e8cefd177"} Jul 06 23:50:43.842281 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:43.842212 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-654cf8646-mttr5" podStartSLOduration=1.8421970509999999 podStartE2EDuration="1.842197051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:43.841362429 +0000 UTC m=+154.104034627" watchObservedRunningTime="2026-07-06 23:50:43.842197051 +0000 UTC m=+154.104869246" Jul 06 23:50:44.124520 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.124427 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 23:50:44.127139 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.127121 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.133139 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.133116 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 23:50:44.133267 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.133185 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-99nqs\"" Jul 06 23:50:44.134077 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.134056 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 23:50:44.134178 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.134079 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 23:50:44.134178 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.134083 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 23:50:44.134178 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.134160 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 23:50:44.134662 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.134644 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 23:50:44.134757 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.134738 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 23:50:44.135046 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.135027 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 23:50:44.140266 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.140249 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 23:50:44.144291 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.144271 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 23:50:44.277377 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277341 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277377 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277378 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/719b2d13-e001-4fde-8c72-6d082d75be25-config-out\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277399 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277421 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/719b2d13-e001-4fde-8c72-6d082d75be25-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277491 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxtcq\" (UniqueName: \"kubernetes.io/projected/719b2d13-e001-4fde-8c72-6d082d75be25-kube-api-access-hxtcq\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277540 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/719b2d13-e001-4fde-8c72-6d082d75be25-tls-assets\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277567 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277592 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/719b2d13-e001-4fde-8c72-6d082d75be25-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277615 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277918 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277672 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277918 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277713 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/719b2d13-e001-4fde-8c72-6d082d75be25-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277918 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277746 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-web-config\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.277918 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.277778 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-config-volume\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378206 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378133 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378206 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378169 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/719b2d13-e001-4fde-8c72-6d082d75be25-config-out\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378206 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378191 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378427 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/719b2d13-e001-4fde-8c72-6d082d75be25-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378427 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378235 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxtcq\" (UniqueName: \"kubernetes.io/projected/719b2d13-e001-4fde-8c72-6d082d75be25-kube-api-access-hxtcq\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378427 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378262 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/719b2d13-e001-4fde-8c72-6d082d75be25-tls-assets\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378427 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378427 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378326 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/719b2d13-e001-4fde-8c72-6d082d75be25-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378427 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378355 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378427 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378380 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378427 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378403 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/719b2d13-e001-4fde-8c72-6d082d75be25-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378427 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378429 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-web-config\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.378892 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.378456 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-config-volume\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.380216 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.379139 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/719b2d13-e001-4fde-8c72-6d082d75be25-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.380216 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.380193 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/719b2d13-e001-4fde-8c72-6d082d75be25-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.380493 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.380456 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/719b2d13-e001-4fde-8c72-6d082d75be25-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.381549 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.381520 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/719b2d13-e001-4fde-8c72-6d082d75be25-config-out\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.381888 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.381864 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.381981 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.381943 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-config-volume\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.382030 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.381995 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.382082 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.381995 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.382302 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.382282 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.382363 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.382310 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/719b2d13-e001-4fde-8c72-6d082d75be25-tls-assets\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.383004 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.382984 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-web-config\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.383896 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.383878 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/719b2d13-e001-4fde-8c72-6d082d75be25-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.386547 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.386528 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxtcq\" (UniqueName: \"kubernetes.io/projected/719b2d13-e001-4fde-8c72-6d082d75be25-kube-api-access-hxtcq\") pod \"alertmanager-main-0\" (UID: \"719b2d13-e001-4fde-8c72-6d082d75be25\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.436935 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.436908 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 23:50:44.571907 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.571802 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 23:50:44.574122 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:50:44.574095 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod719b2d13_e001_4fde_8c72_6d082d75be25.slice/crio-fa2c85e62ca7cfbc8765c04a23b5b345ea8377e00cd856e4b0d77c09f2f15137 WatchSource:0}: Error finding container fa2c85e62ca7cfbc8765c04a23b5b345ea8377e00cd856e4b0d77c09f2f15137: Status 404 returned error can't find the container with id fa2c85e62ca7cfbc8765c04a23b5b345ea8377e00cd856e4b0d77c09f2f15137 Jul 06 23:50:44.830071 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.830038 2583 generic.go:356] "Generic (PLEG): container finished" podID="719b2d13-e001-4fde-8c72-6d082d75be25" containerID="a1d0cc4391d73ca7504124b221876c24ca7c03dc6ee10de3bcf83c9f79d768b4" exitCode=0 Jul 06 23:50:44.830492 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.830129 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"719b2d13-e001-4fde-8c72-6d082d75be25","Type":"ContainerDied","Data":"a1d0cc4391d73ca7504124b221876c24ca7c03dc6ee10de3bcf83c9f79d768b4"} Jul 06 23:50:44.830492 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:44.830417 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"719b2d13-e001-4fde-8c72-6d082d75be25","Type":"ContainerStarted","Data":"fa2c85e62ca7cfbc8765c04a23b5b345ea8377e00cd856e4b0d77c09f2f15137"} Jul 06 23:50:46.843333 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:46.843297 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"719b2d13-e001-4fde-8c72-6d082d75be25","Type":"ContainerStarted","Data":"7b29c524c334f406fa2e28356526000d8308469f600c78bd669ccbe1304ddda2"} Jul 06 23:50:46.843720 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:46.843378 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"719b2d13-e001-4fde-8c72-6d082d75be25","Type":"ContainerStarted","Data":"e930488071c02e691460b8992f304ab08656b065ba9c1ca8129f330a3549de78"} Jul 06 23:50:46.843720 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:46.843418 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"719b2d13-e001-4fde-8c72-6d082d75be25","Type":"ContainerStarted","Data":"a250326922039a90c4c9e60f80c56fb4d5b3aa7d800e54fceea0c592cc89b2b8"} Jul 06 23:50:46.843720 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:46.843454 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"719b2d13-e001-4fde-8c72-6d082d75be25","Type":"ContainerStarted","Data":"0a5de770ca9043d2f355074334d97413ea1ef32419a6288e0b02b4c727704132"} Jul 06 23:50:46.843720 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:46.843505 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"719b2d13-e001-4fde-8c72-6d082d75be25","Type":"ContainerStarted","Data":"a81815577b8694212ce67251d9e5306770282747b4dfa564ddeaa7b19defde33"} Jul 06 23:50:46.843720 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:46.843542 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"719b2d13-e001-4fde-8c72-6d082d75be25","Type":"ContainerStarted","Data":"11d6f098fc0d00c65eb39a9f2591a432050dd5595e4c6ab6e9c13ceec59c7ed6"} Jul 06 23:50:46.872916 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:46.872863 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.649088111 podStartE2EDuration="2.872847285s" totalImagesPullingTime="1.209924004s" totalInitContainerRuntime="13.83517ms" isStatefulPod=false podCreationTimestamp="2026-07-06 23:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:50:46.872141073 +0000 UTC m=+157.134813285" watchObservedRunningTime="2026-07-06 23:50:46.872847285 +0000 UTC m=+157.135519479" Jul 06 23:50:52.840948 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:52.840904 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:52.841413 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:52.841004 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:52.845825 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:52.845802 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:50:52.869744 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:50:52.869718 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:51:25.542155 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.542122 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl"] Jul 06 23:51:25.544591 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.544575 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.546673 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.546640 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 23:51:25.546792 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.546753 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b7d6q\"" Jul 06 23:51:25.547311 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.547298 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 23:51:25.553060 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.553039 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl"] Jul 06 23:51:25.618221 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.618187 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.618221 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.618218 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.618414 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.618293 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf7zt\" (UniqueName: \"kubernetes.io/projected/ac573a65-fb4e-491c-991e-63ce643c2e9f-kube-api-access-cf7zt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.719429 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.719393 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf7zt\" (UniqueName: \"kubernetes.io/projected/ac573a65-fb4e-491c-991e-63ce643c2e9f-kube-api-access-cf7zt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.719591 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.719444 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.719591 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.719460 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.719933 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.719911 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.719975 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.719911 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.727739 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.727715 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf7zt\" (UniqueName: \"kubernetes.io/projected/ac573a65-fb4e-491c-991e-63ce643c2e9f-kube-api-access-cf7zt\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.853682 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.853603 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:25.975646 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.975579 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl"] Jul 06 23:51:25.978596 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:51:25.978564 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac573a65_fb4e_491c_991e_63ce643c2e9f.slice/crio-6da812977082fe8b2d67c7104c2c811100bd956fde8a1c07062749ea69c169bb WatchSource:0}: Error finding container 6da812977082fe8b2d67c7104c2c811100bd956fde8a1c07062749ea69c169bb: Status 404 returned error can't find the container with id 6da812977082fe8b2d67c7104c2c811100bd956fde8a1c07062749ea69c169bb Jul 06 23:51:25.987307 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:25.987280 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" event={"ID":"ac573a65-fb4e-491c-991e-63ce643c2e9f","Type":"ContainerStarted","Data":"6da812977082fe8b2d67c7104c2c811100bd956fde8a1c07062749ea69c169bb"} Jul 06 23:51:33.018914 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:33.018876 2583 generic.go:356] "Generic (PLEG): container finished" podID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerID="5e65944b5ea5d6a2fbe405d801178d3c5943c89f41fb44c78749f30659540645" exitCode=0 Jul 06 23:51:33.019391 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:33.018974 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" event={"ID":"ac573a65-fb4e-491c-991e-63ce643c2e9f","Type":"ContainerDied","Data":"5e65944b5ea5d6a2fbe405d801178d3c5943c89f41fb44c78749f30659540645"} Jul 06 23:51:36.032261 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:36.032226 2583 generic.go:356] "Generic (PLEG): container finished" podID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerID="06fa3ad70be3861ebb73de3546f66f8e3b36a32c9230a40f7b73281a7e0d0f09" exitCode=0 Jul 06 23:51:36.032692 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:36.032285 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" event={"ID":"ac573a65-fb4e-491c-991e-63ce643c2e9f","Type":"ContainerDied","Data":"06fa3ad70be3861ebb73de3546f66f8e3b36a32c9230a40f7b73281a7e0d0f09"} Jul 06 23:51:43.061714 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:43.061631 2583 generic.go:356] "Generic (PLEG): container finished" podID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerID="4a16a4cf57590f391192c450f3c5f260cd197733d9122f5c286c455cd23d5df0" exitCode=0 Jul 06 23:51:43.062049 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:43.061719 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" event={"ID":"ac573a65-fb4e-491c-991e-63ce643c2e9f","Type":"ContainerDied","Data":"4a16a4cf57590f391192c450f3c5f260cd197733d9122f5c286c455cd23d5df0"} Jul 06 23:51:44.187274 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.187247 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:44.284655 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.284623 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf7zt\" (UniqueName: \"kubernetes.io/projected/ac573a65-fb4e-491c-991e-63ce643c2e9f-kube-api-access-cf7zt\") pod \"ac573a65-fb4e-491c-991e-63ce643c2e9f\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " Jul 06 23:51:44.284818 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.284699 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-util\") pod \"ac573a65-fb4e-491c-991e-63ce643c2e9f\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " Jul 06 23:51:44.284818 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.284740 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-bundle\") pod \"ac573a65-fb4e-491c-991e-63ce643c2e9f\" (UID: \"ac573a65-fb4e-491c-991e-63ce643c2e9f\") " Jul 06 23:51:44.285374 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.285345 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-bundle" (OuterVolumeSpecName: "bundle") pod "ac573a65-fb4e-491c-991e-63ce643c2e9f" (UID: "ac573a65-fb4e-491c-991e-63ce643c2e9f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:51:44.286793 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.286762 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac573a65-fb4e-491c-991e-63ce643c2e9f-kube-api-access-cf7zt" (OuterVolumeSpecName: "kube-api-access-cf7zt") pod "ac573a65-fb4e-491c-991e-63ce643c2e9f" (UID: "ac573a65-fb4e-491c-991e-63ce643c2e9f"). InnerVolumeSpecName "kube-api-access-cf7zt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:51:44.289126 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.289107 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-util" (OuterVolumeSpecName: "util") pod "ac573a65-fb4e-491c-991e-63ce643c2e9f" (UID: "ac573a65-fb4e-491c-991e-63ce643c2e9f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:51:44.385370 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.385286 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-bundle\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:51:44.385370 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.385317 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf7zt\" (UniqueName: \"kubernetes.io/projected/ac573a65-fb4e-491c-991e-63ce643c2e9f-kube-api-access-cf7zt\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:51:44.385370 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:44.385327 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac573a65-fb4e-491c-991e-63ce643c2e9f-util\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:51:45.070386 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:45.070360 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" Jul 06 23:51:45.070546 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:45.070359 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rjgcl" event={"ID":"ac573a65-fb4e-491c-991e-63ce643c2e9f","Type":"ContainerDied","Data":"6da812977082fe8b2d67c7104c2c811100bd956fde8a1c07062749ea69c169bb"} Jul 06 23:51:45.070546 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:45.070497 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6da812977082fe8b2d67c7104c2c811100bd956fde8a1c07062749ea69c169bb" Jul 06 23:51:48.493232 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.493200 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx"] Jul 06 23:51:48.493631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.493504 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerName="extract" Jul 06 23:51:48.493631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.493518 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerName="extract" Jul 06 23:51:48.493631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.493529 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerName="pull" Jul 06 23:51:48.493631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.493535 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerName="pull" Jul 06 23:51:48.493631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.493554 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerName="util" Jul 06 23:51:48.493631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.493561 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerName="util" Jul 06 23:51:48.493631 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.493626 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac573a65-fb4e-491c-991e-63ce643c2e9f" containerName="extract" Jul 06 23:51:48.535074 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.535041 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx"] Jul 06 23:51:48.535221 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.535145 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" Jul 06 23:51:48.537441 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.537417 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 23:51:48.537599 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.537506 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:51:48.537599 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.537565 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-5prhf\"" Jul 06 23:51:48.619768 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.619726 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/259ec929-851a-4228-9c74-f1c2be99fd79-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4s9nx\" (UID: \"259ec929-851a-4228-9c74-f1c2be99fd79\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" Jul 06 23:51:48.619940 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.619785 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grhcf\" (UniqueName: \"kubernetes.io/projected/259ec929-851a-4228-9c74-f1c2be99fd79-kube-api-access-grhcf\") pod \"cert-manager-operator-controller-manager-55c444bf86-4s9nx\" (UID: \"259ec929-851a-4228-9c74-f1c2be99fd79\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" Jul 06 23:51:48.720784 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.720744 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grhcf\" (UniqueName: \"kubernetes.io/projected/259ec929-851a-4228-9c74-f1c2be99fd79-kube-api-access-grhcf\") pod \"cert-manager-operator-controller-manager-55c444bf86-4s9nx\" (UID: \"259ec929-851a-4228-9c74-f1c2be99fd79\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" Jul 06 23:51:48.720949 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.720802 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/259ec929-851a-4228-9c74-f1c2be99fd79-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4s9nx\" (UID: \"259ec929-851a-4228-9c74-f1c2be99fd79\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" Jul 06 23:51:48.721112 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.721098 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/259ec929-851a-4228-9c74-f1c2be99fd79-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4s9nx\" (UID: \"259ec929-851a-4228-9c74-f1c2be99fd79\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" Jul 06 23:51:48.728402 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.728377 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grhcf\" (UniqueName: \"kubernetes.io/projected/259ec929-851a-4228-9c74-f1c2be99fd79-kube-api-access-grhcf\") pod \"cert-manager-operator-controller-manager-55c444bf86-4s9nx\" (UID: \"259ec929-851a-4228-9c74-f1c2be99fd79\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" Jul 06 23:51:48.844108 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.844071 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" Jul 06 23:51:48.999118 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:48.999084 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx"] Jul 06 23:51:49.002145 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:51:49.002120 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod259ec929_851a_4228_9c74_f1c2be99fd79.slice/crio-f5fd54b197e3dd3860d2556d999c0c74be168a41ed76a7cd5995f1af5f16b8bd WatchSource:0}: Error finding container f5fd54b197e3dd3860d2556d999c0c74be168a41ed76a7cd5995f1af5f16b8bd: Status 404 returned error can't find the container with id f5fd54b197e3dd3860d2556d999c0c74be168a41ed76a7cd5995f1af5f16b8bd Jul 06 23:51:49.087037 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:49.087001 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" event={"ID":"259ec929-851a-4228-9c74-f1c2be99fd79","Type":"ContainerStarted","Data":"f5fd54b197e3dd3860d2556d999c0c74be168a41ed76a7cd5995f1af5f16b8bd"} Jul 06 23:51:52.104329 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.104291 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" event={"ID":"259ec929-851a-4228-9c74-f1c2be99fd79","Type":"ContainerStarted","Data":"f5dd139dd67c085a4397ae733fbdcd2a82a146ae5610902b5a3a3d9e6ad4afca"} Jul 06 23:51:52.128639 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.128588 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4s9nx" podStartSLOduration=1.9896767629999998 podStartE2EDuration="4.128571089s" totalImagesPullingTime="2.138894326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:51:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:51:52.126923767 +0000 UTC m=+222.389595963" watchObservedRunningTime="2026-07-06 23:51:52.128571089 +0000 UTC m=+222.391243283" Jul 06 23:51:52.836821 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.836788 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4"] Jul 06 23:51:52.839347 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.839331 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:52.841905 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.841882 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b7d6q\"" Jul 06 23:51:52.842042 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.841922 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 23:51:52.847850 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.847830 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 23:51:52.863643 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.863614 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4"] Jul 06 23:51:52.957064 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.957028 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqc55\" (UniqueName: \"kubernetes.io/projected/73dd8d58-292d-429e-96d4-bfdffff7d3ff-kube-api-access-hqc55\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:52.957064 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.957066 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:52.957287 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:52.957141 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:53.057694 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:53.057650 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqc55\" (UniqueName: \"kubernetes.io/projected/73dd8d58-292d-429e-96d4-bfdffff7d3ff-kube-api-access-hqc55\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:53.057694 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:53.057699 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:53.057952 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:53.057758 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:53.058174 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:53.058152 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:53.058244 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:53.058184 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:53.069017 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:53.068989 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqc55\" (UniqueName: \"kubernetes.io/projected/73dd8d58-292d-429e-96d4-bfdffff7d3ff-kube-api-access-hqc55\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:53.149447 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:53.149361 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:51:53.274840 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:53.274816 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4"] Jul 06 23:51:53.276895 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:51:53.276868 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73dd8d58_292d_429e_96d4_bfdffff7d3ff.slice/crio-f633f40f84d676b1bda867c33d1f72a8c24931f196a4f98706883439fc591b2c WatchSource:0}: Error finding container f633f40f84d676b1bda867c33d1f72a8c24931f196a4f98706883439fc591b2c: Status 404 returned error can't find the container with id f633f40f84d676b1bda867c33d1f72a8c24931f196a4f98706883439fc591b2c Jul 06 23:51:54.114004 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:54.113968 2583 generic.go:356] "Generic (PLEG): container finished" podID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerID="65c73b390384a55bd778880289046fb7a23a5acf29f3d295e3db623f04e6ecb2" exitCode=0 Jul 06 23:51:54.114181 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:54.114060 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" event={"ID":"73dd8d58-292d-429e-96d4-bfdffff7d3ff","Type":"ContainerDied","Data":"65c73b390384a55bd778880289046fb7a23a5acf29f3d295e3db623f04e6ecb2"} Jul 06 23:51:54.114181 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:54.114119 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" event={"ID":"73dd8d58-292d-429e-96d4-bfdffff7d3ff","Type":"ContainerStarted","Data":"f633f40f84d676b1bda867c33d1f72a8c24931f196a4f98706883439fc591b2c"} Jul 06 23:51:58.857892 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:58.857848 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n625b"] Jul 06 23:51:58.865489 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:58.865444 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" Jul 06 23:51:58.867886 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:58.867841 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-gf24v\"" Jul 06 23:51:58.868026 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:58.867965 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 23:51:58.868571 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:58.868553 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 23:51:58.868688 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:58.868669 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n625b"] Jul 06 23:51:58.905490 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:58.905437 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83153383-60d3-4887-9a66-3c544f1bd6b7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n625b\" (UID: \"83153383-60d3-4887-9a66-3c544f1bd6b7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" Jul 06 23:51:58.905605 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:58.905587 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs25z\" (UniqueName: \"kubernetes.io/projected/83153383-60d3-4887-9a66-3c544f1bd6b7-kube-api-access-zs25z\") pod \"cert-manager-cainjector-559cdd8c87-n625b\" (UID: \"83153383-60d3-4887-9a66-3c544f1bd6b7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" Jul 06 23:51:59.006394 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:59.006357 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs25z\" (UniqueName: \"kubernetes.io/projected/83153383-60d3-4887-9a66-3c544f1bd6b7-kube-api-access-zs25z\") pod \"cert-manager-cainjector-559cdd8c87-n625b\" (UID: \"83153383-60d3-4887-9a66-3c544f1bd6b7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" Jul 06 23:51:59.006576 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:59.006404 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83153383-60d3-4887-9a66-3c544f1bd6b7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n625b\" (UID: \"83153383-60d3-4887-9a66-3c544f1bd6b7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" Jul 06 23:51:59.013663 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:59.013639 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83153383-60d3-4887-9a66-3c544f1bd6b7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-n625b\" (UID: \"83153383-60d3-4887-9a66-3c544f1bd6b7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" Jul 06 23:51:59.013851 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:59.013832 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs25z\" (UniqueName: \"kubernetes.io/projected/83153383-60d3-4887-9a66-3c544f1bd6b7-kube-api-access-zs25z\") pod \"cert-manager-cainjector-559cdd8c87-n625b\" (UID: \"83153383-60d3-4887-9a66-3c544f1bd6b7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" Jul 06 23:51:59.184803 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:59.184714 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" Jul 06 23:51:59.311408 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:51:59.311376 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-n625b"] Jul 06 23:51:59.313915 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:51:59.313888 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83153383_60d3_4887_9a66_3c544f1bd6b7.slice/crio-2a5f45fc7961920525b044a9e5a5df93d83a6be06f8c9c593be7e19793f36aff WatchSource:0}: Error finding container 2a5f45fc7961920525b044a9e5a5df93d83a6be06f8c9c593be7e19793f36aff: Status 404 returned error can't find the container with id 2a5f45fc7961920525b044a9e5a5df93d83a6be06f8c9c593be7e19793f36aff Jul 06 23:52:00.143260 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:00.143168 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" event={"ID":"83153383-60d3-4887-9a66-3c544f1bd6b7","Type":"ContainerStarted","Data":"2a5f45fc7961920525b044a9e5a5df93d83a6be06f8c9c593be7e19793f36aff"} Jul 06 23:52:02.153619 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:02.153586 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" event={"ID":"83153383-60d3-4887-9a66-3c544f1bd6b7","Type":"ContainerStarted","Data":"8dc03bebe723ef17095c35ceadb80c98db81e451d9e47f3a0d11ab94d6726d97"} Jul 06 23:52:02.168058 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:02.168008 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-n625b" podStartSLOduration=1.741392253 podStartE2EDuration="4.167993063s" totalImagesPullingTime="2.42660081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:51:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:02.167281615 +0000 UTC m=+232.429953824" watchObservedRunningTime="2026-07-06 23:52:02.167993063 +0000 UTC m=+232.430665258" Jul 06 23:52:07.176569 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:07.176529 2583 generic.go:356] "Generic (PLEG): container finished" podID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerID="e626786d685a85f642f2cd731d0fd5c9d7a6a563ad218dbcd9c8ff1827ea3b39" exitCode=0 Jul 06 23:52:07.176961 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:07.176602 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" event={"ID":"73dd8d58-292d-429e-96d4-bfdffff7d3ff","Type":"ContainerDied","Data":"e626786d685a85f642f2cd731d0fd5c9d7a6a563ad218dbcd9c8ff1827ea3b39"} Jul 06 23:52:08.182540 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:08.182504 2583 generic.go:356] "Generic (PLEG): container finished" podID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerID="f24411ed73d2dcfd6a27be60f86fee3dd49386e45a6dbaed49eb77b1db173e5c" exitCode=0 Jul 06 23:52:08.182989 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:08.182563 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" event={"ID":"73dd8d58-292d-429e-96d4-bfdffff7d3ff","Type":"ContainerDied","Data":"f24411ed73d2dcfd6a27be60f86fee3dd49386e45a6dbaed49eb77b1db173e5c"} Jul 06 23:52:09.314950 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.314925 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:52:09.394582 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.394558 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-util\") pod \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " Jul 06 23:52:09.394719 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.394642 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-bundle\") pod \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " Jul 06 23:52:09.394719 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.394682 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqc55\" (UniqueName: \"kubernetes.io/projected/73dd8d58-292d-429e-96d4-bfdffff7d3ff-kube-api-access-hqc55\") pod \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\" (UID: \"73dd8d58-292d-429e-96d4-bfdffff7d3ff\") " Jul 06 23:52:09.395009 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.394976 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-bundle" (OuterVolumeSpecName: "bundle") pod "73dd8d58-292d-429e-96d4-bfdffff7d3ff" (UID: "73dd8d58-292d-429e-96d4-bfdffff7d3ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:52:09.396870 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.396838 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73dd8d58-292d-429e-96d4-bfdffff7d3ff-kube-api-access-hqc55" (OuterVolumeSpecName: "kube-api-access-hqc55") pod "73dd8d58-292d-429e-96d4-bfdffff7d3ff" (UID: "73dd8d58-292d-429e-96d4-bfdffff7d3ff"). InnerVolumeSpecName "kube-api-access-hqc55". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:52:09.399253 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.399235 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-util" (OuterVolumeSpecName: "util") pod "73dd8d58-292d-429e-96d4-bfdffff7d3ff" (UID: "73dd8d58-292d-429e-96d4-bfdffff7d3ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:52:09.495879 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.495853 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-bundle\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:52:09.495879 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.495880 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqc55\" (UniqueName: \"kubernetes.io/projected/73dd8d58-292d-429e-96d4-bfdffff7d3ff-kube-api-access-hqc55\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:52:09.496042 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:09.495890 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/73dd8d58-292d-429e-96d4-bfdffff7d3ff-util\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:52:10.192581 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:10.192548 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" event={"ID":"73dd8d58-292d-429e-96d4-bfdffff7d3ff","Type":"ContainerDied","Data":"f633f40f84d676b1bda867c33d1f72a8c24931f196a4f98706883439fc591b2c"} Jul 06 23:52:10.192581 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:10.192587 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f633f40f84d676b1bda867c33d1f72a8c24931f196a4f98706883439fc591b2c" Jul 06 23:52:10.192808 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:10.192604 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flt8p4" Jul 06 23:52:12.557497 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.557448 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-42cc6"] Jul 06 23:52:12.557869 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.557770 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerName="pull" Jul 06 23:52:12.557869 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.557781 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerName="pull" Jul 06 23:52:12.557869 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.557791 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerName="extract" Jul 06 23:52:12.557869 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.557796 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerName="extract" Jul 06 23:52:12.557869 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.557809 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerName="util" Jul 06 23:52:12.557869 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.557815 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerName="util" Jul 06 23:52:12.557869 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.557863 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="73dd8d58-292d-429e-96d4-bfdffff7d3ff" containerName="extract" Jul 06 23:52:12.561622 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.561606 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-42cc6" Jul 06 23:52:12.563512 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.563489 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-ktfcz\"" Jul 06 23:52:12.567879 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.567855 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-42cc6"] Jul 06 23:52:12.621135 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.621105 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06312f2c-dd62-461d-8f0a-88efefa9ffcb-bound-sa-token\") pod \"cert-manager-55b4d954d5-42cc6\" (UID: \"06312f2c-dd62-461d-8f0a-88efefa9ffcb\") " pod="cert-manager/cert-manager-55b4d954d5-42cc6" Jul 06 23:52:12.621135 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.621140 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl67w\" (UniqueName: \"kubernetes.io/projected/06312f2c-dd62-461d-8f0a-88efefa9ffcb-kube-api-access-rl67w\") pod \"cert-manager-55b4d954d5-42cc6\" (UID: \"06312f2c-dd62-461d-8f0a-88efefa9ffcb\") " pod="cert-manager/cert-manager-55b4d954d5-42cc6" Jul 06 23:52:12.722040 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.722008 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06312f2c-dd62-461d-8f0a-88efefa9ffcb-bound-sa-token\") pod \"cert-manager-55b4d954d5-42cc6\" (UID: \"06312f2c-dd62-461d-8f0a-88efefa9ffcb\") " pod="cert-manager/cert-manager-55b4d954d5-42cc6" Jul 06 23:52:12.722040 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.722046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl67w\" (UniqueName: \"kubernetes.io/projected/06312f2c-dd62-461d-8f0a-88efefa9ffcb-kube-api-access-rl67w\") pod \"cert-manager-55b4d954d5-42cc6\" (UID: \"06312f2c-dd62-461d-8f0a-88efefa9ffcb\") " pod="cert-manager/cert-manager-55b4d954d5-42cc6" Jul 06 23:52:12.729672 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.729640 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/06312f2c-dd62-461d-8f0a-88efefa9ffcb-bound-sa-token\") pod \"cert-manager-55b4d954d5-42cc6\" (UID: \"06312f2c-dd62-461d-8f0a-88efefa9ffcb\") " pod="cert-manager/cert-manager-55b4d954d5-42cc6" Jul 06 23:52:12.729787 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.729731 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl67w\" (UniqueName: \"kubernetes.io/projected/06312f2c-dd62-461d-8f0a-88efefa9ffcb-kube-api-access-rl67w\") pod \"cert-manager-55b4d954d5-42cc6\" (UID: \"06312f2c-dd62-461d-8f0a-88efefa9ffcb\") " pod="cert-manager/cert-manager-55b4d954d5-42cc6" Jul 06 23:52:12.870818 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:12.870723 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-42cc6" Jul 06 23:52:13.017257 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:13.017098 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-42cc6"] Jul 06 23:52:13.019810 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:52:13.019783 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06312f2c_dd62_461d_8f0a_88efefa9ffcb.slice/crio-9b86f55368721671c94ac43bf9514915f9d781a42b64c1ffd3292a4aded800e7 WatchSource:0}: Error finding container 9b86f55368721671c94ac43bf9514915f9d781a42b64c1ffd3292a4aded800e7: Status 404 returned error can't find the container with id 9b86f55368721671c94ac43bf9514915f9d781a42b64c1ffd3292a4aded800e7 Jul 06 23:52:13.205755 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:13.205713 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-42cc6" event={"ID":"06312f2c-dd62-461d-8f0a-88efefa9ffcb","Type":"ContainerStarted","Data":"97c8fe94304385fb60a3bdf4d62b0295e554c383decbadf95789ca6fadc76717"} Jul 06 23:52:13.205755 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:13.205759 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-42cc6" event={"ID":"06312f2c-dd62-461d-8f0a-88efefa9ffcb","Type":"ContainerStarted","Data":"9b86f55368721671c94ac43bf9514915f9d781a42b64c1ffd3292a4aded800e7"} Jul 06 23:52:13.219688 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:13.219639 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-42cc6" podStartSLOduration=1.219624153 podStartE2EDuration="1.219624153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:13.218992751 +0000 UTC m=+243.481664983" watchObservedRunningTime="2026-07-06 23:52:13.219624153 +0000 UTC m=+243.482296326" Jul 06 23:52:16.973933 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:16.973895 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g"] Jul 06 23:52:16.976062 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:16.976046 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" Jul 06 23:52:16.978013 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:16.977992 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 23:52:16.978013 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:16.978005 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 23:52:16.978584 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:16.978560 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-dk47k\"" Jul 06 23:52:16.983923 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:16.983900 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g"] Jul 06 23:52:17.058795 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:17.058758 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/085bfed8-c216-4027-8809-981b273dd98d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wrc7g\" (UID: \"085bfed8-c216-4027-8809-981b273dd98d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" Jul 06 23:52:17.058967 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:17.058826 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rtwm\" (UniqueName: \"kubernetes.io/projected/085bfed8-c216-4027-8809-981b273dd98d-kube-api-access-7rtwm\") pod \"openshift-lws-operator-fd8ccff4c-wrc7g\" (UID: \"085bfed8-c216-4027-8809-981b273dd98d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" Jul 06 23:52:17.160049 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:17.160014 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rtwm\" (UniqueName: \"kubernetes.io/projected/085bfed8-c216-4027-8809-981b273dd98d-kube-api-access-7rtwm\") pod \"openshift-lws-operator-fd8ccff4c-wrc7g\" (UID: \"085bfed8-c216-4027-8809-981b273dd98d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" Jul 06 23:52:17.160187 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:17.160091 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/085bfed8-c216-4027-8809-981b273dd98d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wrc7g\" (UID: \"085bfed8-c216-4027-8809-981b273dd98d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" Jul 06 23:52:17.160494 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:17.160452 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/085bfed8-c216-4027-8809-981b273dd98d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wrc7g\" (UID: \"085bfed8-c216-4027-8809-981b273dd98d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" Jul 06 23:52:17.167643 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:17.167607 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rtwm\" (UniqueName: \"kubernetes.io/projected/085bfed8-c216-4027-8809-981b273dd98d-kube-api-access-7rtwm\") pod \"openshift-lws-operator-fd8ccff4c-wrc7g\" (UID: \"085bfed8-c216-4027-8809-981b273dd98d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" Jul 06 23:52:17.285439 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:17.285406 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" Jul 06 23:52:17.407896 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:17.407872 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g"] Jul 06 23:52:17.410552 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:52:17.410523 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod085bfed8_c216_4027_8809_981b273dd98d.slice/crio-fb2f9bf71d336e2b6d00bd664e952061761f046433735ec17c14c232693730ca WatchSource:0}: Error finding container fb2f9bf71d336e2b6d00bd664e952061761f046433735ec17c14c232693730ca: Status 404 returned error can't find the container with id fb2f9bf71d336e2b6d00bd664e952061761f046433735ec17c14c232693730ca Jul 06 23:52:18.230543 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:18.230509 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" event={"ID":"085bfed8-c216-4027-8809-981b273dd98d","Type":"ContainerStarted","Data":"fb2f9bf71d336e2b6d00bd664e952061761f046433735ec17c14c232693730ca"} Jul 06 23:52:19.236468 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:19.236429 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" event={"ID":"085bfed8-c216-4027-8809-981b273dd98d","Type":"ContainerStarted","Data":"f26df58214cca29f66486c5fd9c8b4056353f0a67a3ac1c3adc92c9969f6188d"} Jul 06 23:52:19.255358 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:19.255252 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wrc7g" podStartSLOduration=1.663553056 podStartE2EDuration="3.255238048s" totalImagesPullingTime="1.591684992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:19.253787388 +0000 UTC m=+249.516459583" watchObservedRunningTime="2026-07-06 23:52:19.255238048 +0000 UTC m=+249.517910243" Jul 06 23:52:23.005653 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.005614 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2"] Jul 06 23:52:23.007855 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.007835 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.009924 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.009899 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 23:52:23.010045 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.010028 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 23:52:23.010691 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.010672 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-b7d6q\"" Jul 06 23:52:23.017086 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.017066 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2"] Jul 06 23:52:23.114542 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.114509 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdjfl\" (UniqueName: \"kubernetes.io/projected/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-kube-api-access-qdjfl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.114674 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.114557 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.114674 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.114630 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.215353 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.215325 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.215502 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.215378 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.215502 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.215405 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdjfl\" (UniqueName: \"kubernetes.io/projected/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-kube-api-access-qdjfl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.215743 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.215726 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.215782 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.215754 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.222820 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.222797 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdjfl\" (UniqueName: \"kubernetes.io/projected/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-kube-api-access-qdjfl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.317513 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.317431 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:23.439371 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:23.439350 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2"] Jul 06 23:52:23.441684 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:52:23.441660 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb21bc39_a0a6_4a13_80c3_11568d3da8f1.slice/crio-5b8b4f7109dc51793d836d4e8d41421c942244d807c1bd1adc167a2f5a28445d WatchSource:0}: Error finding container 5b8b4f7109dc51793d836d4e8d41421c942244d807c1bd1adc167a2f5a28445d: Status 404 returned error can't find the container with id 5b8b4f7109dc51793d836d4e8d41421c942244d807c1bd1adc167a2f5a28445d Jul 06 23:52:24.258139 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:24.258108 2583 generic.go:356] "Generic (PLEG): container finished" podID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerID="770b2ca65fe61f7a0690ba34f790bd52d9dcfb4de8c026db718ac5dfeabcfb48" exitCode=0 Jul 06 23:52:24.258517 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:24.258148 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" event={"ID":"cb21bc39-a0a6-4a13-80c3-11568d3da8f1","Type":"ContainerDied","Data":"770b2ca65fe61f7a0690ba34f790bd52d9dcfb4de8c026db718ac5dfeabcfb48"} Jul 06 23:52:24.258517 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:24.258210 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" event={"ID":"cb21bc39-a0a6-4a13-80c3-11568d3da8f1","Type":"ContainerStarted","Data":"5b8b4f7109dc51793d836d4e8d41421c942244d807c1bd1adc167a2f5a28445d"} Jul 06 23:52:25.263669 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:25.263620 2583 generic.go:356] "Generic (PLEG): container finished" podID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerID="58ea17d0350adeaa584b4d12cd74eaf5b3da58593a7f3fdb1ef41b036f350197" exitCode=0 Jul 06 23:52:25.264058 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:25.263694 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" event={"ID":"cb21bc39-a0a6-4a13-80c3-11568d3da8f1","Type":"ContainerDied","Data":"58ea17d0350adeaa584b4d12cd74eaf5b3da58593a7f3fdb1ef41b036f350197"} Jul 06 23:52:26.269780 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:26.269742 2583 generic.go:356] "Generic (PLEG): container finished" podID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerID="b8aad00c9d543b694f1fc7d86bbb089319fda7e3f1565141e375c377441bf7c9" exitCode=0 Jul 06 23:52:26.270141 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:26.269787 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" event={"ID":"cb21bc39-a0a6-4a13-80c3-11568d3da8f1","Type":"ContainerDied","Data":"b8aad00c9d543b694f1fc7d86bbb089319fda7e3f1565141e375c377441bf7c9"} Jul 06 23:52:27.394938 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.394916 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:27.556722 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.556626 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdjfl\" (UniqueName: \"kubernetes.io/projected/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-kube-api-access-qdjfl\") pod \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " Jul 06 23:52:27.556722 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.556675 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-util\") pod \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " Jul 06 23:52:27.556722 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.556706 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-bundle\") pod \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\" (UID: \"cb21bc39-a0a6-4a13-80c3-11568d3da8f1\") " Jul 06 23:52:27.557429 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.557404 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-bundle" (OuterVolumeSpecName: "bundle") pod "cb21bc39-a0a6-4a13-80c3-11568d3da8f1" (UID: "cb21bc39-a0a6-4a13-80c3-11568d3da8f1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:52:27.558743 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.558722 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-kube-api-access-qdjfl" (OuterVolumeSpecName: "kube-api-access-qdjfl") pod "cb21bc39-a0a6-4a13-80c3-11568d3da8f1" (UID: "cb21bc39-a0a6-4a13-80c3-11568d3da8f1"). InnerVolumeSpecName "kube-api-access-qdjfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:52:27.561873 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.561838 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-util" (OuterVolumeSpecName: "util") pod "cb21bc39-a0a6-4a13-80c3-11568d3da8f1" (UID: "cb21bc39-a0a6-4a13-80c3-11568d3da8f1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:52:27.657392 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.657357 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-bundle\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:52:27.657392 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.657384 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdjfl\" (UniqueName: \"kubernetes.io/projected/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-kube-api-access-qdjfl\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:52:27.657392 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:27.657394 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cb21bc39-a0a6-4a13-80c3-11568d3da8f1-util\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:52:28.280200 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:28.280171 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" event={"ID":"cb21bc39-a0a6-4a13-80c3-11568d3da8f1","Type":"ContainerDied","Data":"5b8b4f7109dc51793d836d4e8d41421c942244d807c1bd1adc167a2f5a28445d"} Jul 06 23:52:28.280200 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:28.280212 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b8b4f7109dc51793d836d4e8d41421c942244d807c1bd1adc167a2f5a28445d" Jul 06 23:52:28.280410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:28.280213 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4vnc2" Jul 06 23:52:35.727996 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.727905 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8"] Jul 06 23:52:35.728384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.728228 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerName="util" Jul 06 23:52:35.728384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.728240 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerName="util" Jul 06 23:52:35.728384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.728261 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerName="extract" Jul 06 23:52:35.728384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.728266 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerName="extract" Jul 06 23:52:35.728384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.728277 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerName="pull" Jul 06 23:52:35.728384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.728284 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerName="pull" Jul 06 23:52:35.728384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.728338 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb21bc39-a0a6-4a13-80c3-11568d3da8f1" containerName="extract" Jul 06 23:52:35.730616 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.730601 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.732928 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.732901 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 23:52:35.733065 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.732944 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 23:52:35.733065 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.732966 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 23:52:35.733216 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.733199 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 23:52:35.733282 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.733234 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kmgzr\"" Jul 06 23:52:35.754912 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.754882 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8"] Jul 06 23:52:35.820620 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.820583 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbe40b23-970e-4072-a3a6-78985342407b-tmp\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.820803 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.820639 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbe40b23-970e-4072-a3a6-78985342407b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.820803 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.820731 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cbe40b23-970e-4072-a3a6-78985342407b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.820803 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.820789 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cbe40b23-970e-4072-a3a6-78985342407b-manifests\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.820905 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.820807 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4lpr\" (UniqueName: \"kubernetes.io/projected/cbe40b23-970e-4072-a3a6-78985342407b-kube-api-access-m4lpr\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.921166 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.921131 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbe40b23-970e-4072-a3a6-78985342407b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.921329 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.921181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cbe40b23-970e-4072-a3a6-78985342407b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.921329 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.921221 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cbe40b23-970e-4072-a3a6-78985342407b-manifests\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.921329 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.921238 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lpr\" (UniqueName: \"kubernetes.io/projected/cbe40b23-970e-4072-a3a6-78985342407b-kube-api-access-m4lpr\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.921329 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.921266 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbe40b23-970e-4072-a3a6-78985342407b-tmp\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.921717 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.921696 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cbe40b23-970e-4072-a3a6-78985342407b-manifests\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.923447 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.923429 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cbe40b23-970e-4072-a3a6-78985342407b-tmp\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.923702 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.923682 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cbe40b23-970e-4072-a3a6-78985342407b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.923765 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.923726 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cbe40b23-970e-4072-a3a6-78985342407b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:35.929316 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:35.929284 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4lpr\" (UniqueName: \"kubernetes.io/projected/cbe40b23-970e-4072-a3a6-78985342407b-kube-api-access-m4lpr\") pod \"opendatahub-operator-controller-manager-6c4d578756-r2dx8\" (UID: \"cbe40b23-970e-4072-a3a6-78985342407b\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:36.041434 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:36.041399 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:36.173911 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:36.173877 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8"] Jul 06 23:52:36.178416 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:52:36.178383 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbe40b23_970e_4072_a3a6_78985342407b.slice/crio-aec32c766a2862c233e010ce79d2a01f50bb23738176a6a4ea561995166e3abf WatchSource:0}: Error finding container aec32c766a2862c233e010ce79d2a01f50bb23738176a6a4ea561995166e3abf: Status 404 returned error can't find the container with id aec32c766a2862c233e010ce79d2a01f50bb23738176a6a4ea561995166e3abf Jul 06 23:52:36.318504 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:36.318404 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" event={"ID":"cbe40b23-970e-4072-a3a6-78985342407b","Type":"ContainerStarted","Data":"aec32c766a2862c233e010ce79d2a01f50bb23738176a6a4ea561995166e3abf"} Jul 06 23:52:39.336881 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:39.336847 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" event={"ID":"cbe40b23-970e-4072-a3a6-78985342407b","Type":"ContainerStarted","Data":"8340007efb1150a627a6d38a0864c2a33797c6350638e12b0aa7d02edba158b6"} Jul 06 23:52:40.343102 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:40.343066 2583 generic.go:356] "Generic (PLEG): container finished" podID="cbe40b23-970e-4072-a3a6-78985342407b" containerID="8340007efb1150a627a6d38a0864c2a33797c6350638e12b0aa7d02edba158b6" exitCode=0 Jul 06 23:52:40.343498 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:40.343106 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" event={"ID":"cbe40b23-970e-4072-a3a6-78985342407b","Type":"ContainerDied","Data":"8340007efb1150a627a6d38a0864c2a33797c6350638e12b0aa7d02edba158b6"} Jul 06 23:52:41.350889 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:41.350853 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" event={"ID":"cbe40b23-970e-4072-a3a6-78985342407b","Type":"ContainerStarted","Data":"11c613beda14afb6fd776238a34d478f1c3743a1101da610ae554bc16da5d1d9"} Jul 06 23:52:41.351352 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:41.350992 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:41.396753 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:41.396603 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" podStartSLOduration=2.070975106 podStartE2EDuration="6.396583734s" totalImagesPullingTime="3.262082813s" totalInitContainerRuntime="1.063525815s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:52:41.394933475 +0000 UTC m=+271.657605669" watchObservedRunningTime="2026-07-06 23:52:41.396583734 +0000 UTC m=+271.659255931" Jul 06 23:52:52.357795 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:52.357760 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4d578756-r2dx8" Jul 06 23:52:54.699718 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.699682 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb"] Jul 06 23:52:54.705949 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.705922 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:54.713751 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.713717 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 23:52:54.714013 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.713997 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 23:52:54.714286 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.714267 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 23:52:54.714973 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.714953 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 23:52:54.729853 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.729827 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb"] Jul 06 23:52:54.783697 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.783668 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h66k9\" (UniqueName: \"kubernetes.io/projected/ece0863e-d2c2-4310-b949-0bcc0e3279b0-kube-api-access-h66k9\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:54.783863 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.783737 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece0863e-d2c2-4310-b949-0bcc0e3279b0-tmp\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:54.783863 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.783786 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece0863e-d2c2-4310-b949-0bcc0e3279b0-tls-certs\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:54.885020 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.884983 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece0863e-d2c2-4310-b949-0bcc0e3279b0-tls-certs\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:54.885227 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.885052 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h66k9\" (UniqueName: \"kubernetes.io/projected/ece0863e-d2c2-4310-b949-0bcc0e3279b0-kube-api-access-h66k9\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:54.885227 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.885130 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece0863e-d2c2-4310-b949-0bcc0e3279b0-tmp\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:54.885227 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:52:54.885179 2583 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 06 23:52:54.885355 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:52:54.885254 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ece0863e-d2c2-4310-b949-0bcc0e3279b0-tls-certs podName:ece0863e-d2c2-4310-b949-0bcc0e3279b0 nodeName:}" failed. No retries permitted until 2026-07-06 23:52:55.385230651 +0000 UTC m=+285.647902830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ece0863e-d2c2-4310-b949-0bcc0e3279b0-tls-certs") pod "kube-auth-proxy-6c4984cb9-8fgfb" (UID: "ece0863e-d2c2-4310-b949-0bcc0e3279b0") : secret "kube-auth-proxy-tls" not found Jul 06 23:52:54.887341 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.887309 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece0863e-d2c2-4310-b949-0bcc0e3279b0-tmp\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:54.900166 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:54.900136 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h66k9\" (UniqueName: \"kubernetes.io/projected/ece0863e-d2c2-4310-b949-0bcc0e3279b0-kube-api-access-h66k9\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:55.039840 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.039798 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tp54k"] Jul 06 23:52:55.044508 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.044466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:52:55.048759 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.048736 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-2vvc5\"" Jul 06 23:52:55.049150 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.049133 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 23:52:55.059193 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.059170 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tp54k"] Jul 06 23:52:55.086908 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.086870 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d-cert\") pod \"odh-model-controller-7998c67dc4-tp54k\" (UID: \"59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:52:55.087118 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.086947 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xdqx\" (UniqueName: \"kubernetes.io/projected/59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d-kube-api-access-8xdqx\") pod \"odh-model-controller-7998c67dc4-tp54k\" (UID: \"59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:52:55.187387 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.187355 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xdqx\" (UniqueName: \"kubernetes.io/projected/59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d-kube-api-access-8xdqx\") pod \"odh-model-controller-7998c67dc4-tp54k\" (UID: \"59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:52:55.187568 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.187451 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d-cert\") pod \"odh-model-controller-7998c67dc4-tp54k\" (UID: \"59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:52:55.187639 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:52:55.187603 2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 23:52:55.187698 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:52:55.187657 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d-cert podName:59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d nodeName:}" failed. No retries permitted until 2026-07-06 23:52:55.687637874 +0000 UTC m=+285.950310047 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d-cert") pod "odh-model-controller-7998c67dc4-tp54k" (UID: "59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d") : secret "odh-model-controller-webhook-cert" not found Jul 06 23:52:55.198326 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.198294 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xdqx\" (UniqueName: \"kubernetes.io/projected/59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d-kube-api-access-8xdqx\") pod \"odh-model-controller-7998c67dc4-tp54k\" (UID: \"59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:52:55.389595 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.389510 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece0863e-d2c2-4310-b949-0bcc0e3279b0-tls-certs\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:55.391901 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.391873 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece0863e-d2c2-4310-b949-0bcc0e3279b0-tls-certs\") pod \"kube-auth-proxy-6c4984cb9-8fgfb\" (UID: \"ece0863e-d2c2-4310-b949-0bcc0e3279b0\") " pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:55.615386 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.615355 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" Jul 06 23:52:55.692741 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.692706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d-cert\") pod \"odh-model-controller-7998c67dc4-tp54k\" (UID: \"59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:52:55.695384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.695332 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d-cert\") pod \"odh-model-controller-7998c67dc4-tp54k\" (UID: \"59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d\") " pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:52:55.762326 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.762291 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb"] Jul 06 23:52:55.763908 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:52:55.763886 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece0863e_d2c2_4310_b949_0bcc0e3279b0.slice/crio-6f4b8010088e6fbc5b31fa8e1b0702bfe4fedaa96877fce121d87550d3cddb4e WatchSource:0}: Error finding container 6f4b8010088e6fbc5b31fa8e1b0702bfe4fedaa96877fce121d87550d3cddb4e: Status 404 returned error can't find the container with id 6f4b8010088e6fbc5b31fa8e1b0702bfe4fedaa96877fce121d87550d3cddb4e Jul 06 23:52:55.955230 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:55.955136 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:52:56.017952 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.017870 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-b456b"] Jul 06 23:52:56.024592 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.022927 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:52:56.025991 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.025973 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 23:52:56.039886 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.039859 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-b456b"] Jul 06 23:52:56.096054 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.096018 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8414f226-13c8-429a-8a7c-c77150cc7cb1-tls-certs\") pod \"model-serving-api-54d89bd5fc-b456b\" (UID: \"8414f226-13c8-429a-8a7c-c77150cc7cb1\") " pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:52:56.096226 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.096089 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg8wg\" (UniqueName: \"kubernetes.io/projected/8414f226-13c8-429a-8a7c-c77150cc7cb1-kube-api-access-kg8wg\") pod \"model-serving-api-54d89bd5fc-b456b\" (UID: \"8414f226-13c8-429a-8a7c-c77150cc7cb1\") " pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:52:56.104675 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.104646 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nggsp"] Jul 06 23:52:56.107999 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.107979 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.109969 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.109949 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 23:52:56.110108 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.110091 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 23:52:56.110287 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.110259 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 23:52:56.110287 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.110277 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 23:52:56.110419 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.110403 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-42nsv\"" Jul 06 23:52:56.121064 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.121047 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-tp54k"] Jul 06 23:52:56.124119 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:52:56.124094 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59d9a557_9bc9_4a7d_93e0_2b43f44a0d9d.slice/crio-48c599a40c5c4b429e8f480eab334b4889719cf0ad14f00cb11d0b276dffb007 WatchSource:0}: Error finding container 48c599a40c5c4b429e8f480eab334b4889719cf0ad14f00cb11d0b276dffb007: Status 404 returned error can't find the container with id 48c599a40c5c4b429e8f480eab334b4889719cf0ad14f00cb11d0b276dffb007 Jul 06 23:52:56.126040 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.126018 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nggsp"] Jul 06 23:52:56.197105 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.197069 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.197105 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.197106 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-token\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.197337 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.197122 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.197337 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.197196 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8414f226-13c8-429a-8a7c-c77150cc7cb1-tls-certs\") pod \"model-serving-api-54d89bd5fc-b456b\" (UID: \"8414f226-13c8-429a-8a7c-c77150cc7cb1\") " pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:52:56.197337 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:52:56.197269 2583 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 06 23:52:56.197337 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.197297 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.197337 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:52:56.197314 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8414f226-13c8-429a-8a7c-c77150cc7cb1-tls-certs podName:8414f226-13c8-429a-8a7c-c77150cc7cb1 nodeName:}" failed. No retries permitted until 2026-07-06 23:52:56.697299218 +0000 UTC m=+286.959971392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8414f226-13c8-429a-8a7c-c77150cc7cb1-tls-certs") pod "model-serving-api-54d89bd5fc-b456b" (UID: "8414f226-13c8-429a-8a7c-c77150cc7cb1") : secret "model-serving-api-tls" not found Jul 06 23:52:56.197624 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.197410 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg8wg\" (UniqueName: \"kubernetes.io/projected/8414f226-13c8-429a-8a7c-c77150cc7cb1-kube-api-access-kg8wg\") pod \"model-serving-api-54d89bd5fc-b456b\" (UID: \"8414f226-13c8-429a-8a7c-c77150cc7cb1\") " pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:52:56.197624 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.197455 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba485b2-2db2-4647-ade3-a2c54ae4999e-local-certs\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.197624 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.197517 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-cacerts\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.197624 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.197545 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82sgg\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-kube-api-access-82sgg\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.217688 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.217616 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg8wg\" (UniqueName: \"kubernetes.io/projected/8414f226-13c8-429a-8a7c-c77150cc7cb1-kube-api-access-kg8wg\") pod \"model-serving-api-54d89bd5fc-b456b\" (UID: \"8414f226-13c8-429a-8a7c-c77150cc7cb1\") " pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:52:56.298222 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.298181 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.298424 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.298238 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba485b2-2db2-4647-ade3-a2c54ae4999e-local-certs\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.298424 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.298264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-cacerts\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.298424 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.298289 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82sgg\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-kube-api-access-82sgg\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.298424 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.298335 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.298424 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.298359 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-token\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.298424 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.298381 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.299017 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.298992 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.300698 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.300681 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-cacerts\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.300783 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.300724 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.300936 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.300918 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba485b2-2db2-4647-ade3-a2c54ae4999e-local-certs\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.300971 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.300955 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.325207 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.325175 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-token\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.328346 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.328326 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82sgg\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-kube-api-access-82sgg\") pod \"istiod-openshift-gateway-7574459656-nggsp\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.416752 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.416714 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:52:56.427541 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.427511 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" event={"ID":"ece0863e-d2c2-4310-b949-0bcc0e3279b0","Type":"ContainerStarted","Data":"6f4b8010088e6fbc5b31fa8e1b0702bfe4fedaa96877fce121d87550d3cddb4e"} Jul 06 23:52:56.428960 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.428928 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" event={"ID":"59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d","Type":"ContainerStarted","Data":"48c599a40c5c4b429e8f480eab334b4889719cf0ad14f00cb11d0b276dffb007"} Jul 06 23:52:56.583357 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.583326 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nggsp"] Jul 06 23:52:56.585753 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:52:56.585713 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba485b2_2db2_4647_ade3_a2c54ae4999e.slice/crio-200419e71239bb009f9fb625bc1593ab26987bc02fce20a6f966a45bd2090a4d WatchSource:0}: Error finding container 200419e71239bb009f9fb625bc1593ab26987bc02fce20a6f966a45bd2090a4d: Status 404 returned error can't find the container with id 200419e71239bb009f9fb625bc1593ab26987bc02fce20a6f966a45bd2090a4d Jul 06 23:52:56.703232 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.703198 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8414f226-13c8-429a-8a7c-c77150cc7cb1-tls-certs\") pod \"model-serving-api-54d89bd5fc-b456b\" (UID: \"8414f226-13c8-429a-8a7c-c77150cc7cb1\") " pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:52:56.706071 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.706043 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8414f226-13c8-429a-8a7c-c77150cc7cb1-tls-certs\") pod \"model-serving-api-54d89bd5fc-b456b\" (UID: \"8414f226-13c8-429a-8a7c-c77150cc7cb1\") " pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:52:56.937911 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:56.937611 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:52:57.147464 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:57.147439 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-b456b"] Jul 06 23:52:57.440046 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:57.438940 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" event={"ID":"3ba485b2-2db2-4647-ade3-a2c54ae4999e","Type":"ContainerStarted","Data":"200419e71239bb009f9fb625bc1593ab26987bc02fce20a6f966a45bd2090a4d"} Jul 06 23:52:57.441273 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:52:57.441211 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-b456b" event={"ID":"8414f226-13c8-429a-8a7c-c77150cc7cb1","Type":"ContainerStarted","Data":"8799f1c30b241280cc80de43f02af7bd72a10f19f734b1ffb1dee7664aeecd27"} Jul 06 23:53:01.422665 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:01.422623 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:53:01.422977 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:01.422711 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:53:02.473759 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.473714 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" event={"ID":"59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d","Type":"ContainerStarted","Data":"ee58b64d2d6bc14e886fabca214761224a33f78648c77f6632f44f12fd7888a7"} Jul 06 23:53:02.474210 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.473946 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:53:02.475648 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.475576 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" event={"ID":"3ba485b2-2db2-4647-ade3-a2c54ae4999e","Type":"ContainerStarted","Data":"3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea"} Jul 06 23:53:02.477446 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.477422 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-b456b" event={"ID":"8414f226-13c8-429a-8a7c-c77150cc7cb1","Type":"ContainerStarted","Data":"44923d67bbdde3d33df058a3aded1b8c1c6c441007ba8636dc61f1270fb44773"} Jul 06 23:53:02.477659 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.477640 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:53:02.479592 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.479567 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" event={"ID":"ece0863e-d2c2-4310-b949-0bcc0e3279b0","Type":"ContainerStarted","Data":"4047cc4af726329ed01e3cfd37c2552a433d28e36d786b195bad61c0ca1a3e2c"} Jul 06 23:53:02.492782 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.492739 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" podStartSLOduration=2.139140305 podStartE2EDuration="7.492726588s" totalImagesPullingTime="5.353586283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:02.490902446 +0000 UTC m=+292.753574642" watchObservedRunningTime="2026-07-06 23:53:02.492726588 +0000 UTC m=+292.755398783" Jul 06 23:53:02.513532 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.513486 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" podStartSLOduration=1.679694276 podStartE2EDuration="6.513456002s" totalImagesPullingTime="4.833761726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:02.511949498 +0000 UTC m=+292.774621694" watchObservedRunningTime="2026-07-06 23:53:02.513456002 +0000 UTC m=+292.776128187" Jul 06 23:53:02.529780 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.529736 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-b456b" podStartSLOduration=3.266401402 podStartE2EDuration="7.529722338s" totalImagesPullingTime="4.263320936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:02.528533341 +0000 UTC m=+292.791205533" watchObservedRunningTime="2026-07-06 23:53:02.529722338 +0000 UTC m=+292.792394533" Jul 06 23:53:02.552350 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:02.552296 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6c4984cb9-8fgfb" podStartSLOduration=2.901541682 podStartE2EDuration="8.552266921s" totalImagesPullingTime="5.650725239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:52:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:02.5496299 +0000 UTC m=+292.812302102" watchObservedRunningTime="2026-07-06 23:53:02.552266921 +0000 UTC m=+292.814939121" Jul 06 23:53:03.238493 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.238435 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7"] Jul 06 23:53:03.242375 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.242350 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.244627 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.244607 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-z8xrt\"" Jul 06 23:53:03.254664 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.254643 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7"] Jul 06 23:53:03.369574 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.369539 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.369574 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.369578 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a24e7fdb-20db-4a53-83e8-caf0225145d1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.369817 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.369602 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktt8z\" (UniqueName: \"kubernetes.io/projected/a24e7fdb-20db-4a53-83e8-caf0225145d1-kube-api-access-ktt8z\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.369817 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.369718 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.369817 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.369764 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.369932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.369848 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.369932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.369873 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.369932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.369901 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.369932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.369927 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.471541 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471501 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.471747 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471560 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a24e7fdb-20db-4a53-83e8-caf0225145d1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.471747 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471581 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktt8z\" (UniqueName: \"kubernetes.io/projected/a24e7fdb-20db-4a53-83e8-caf0225145d1-kube-api-access-ktt8z\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.471747 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.471747 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471672 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.471747 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471705 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.471747 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471733 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.472056 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471883 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.472056 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.472056 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.471964 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.472231 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.472113 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.472231 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.472141 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.472352 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.472330 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.472643 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.472626 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a24e7fdb-20db-4a53-83e8-caf0225145d1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.474112 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.474084 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.474420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.474186 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.479759 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.479734 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a24e7fdb-20db-4a53-83e8-caf0225145d1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.480130 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.480114 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktt8z\" (UniqueName: \"kubernetes.io/projected/a24e7fdb-20db-4a53-83e8-caf0225145d1-kube-api-access-ktt8z\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t24l7\" (UID: \"a24e7fdb-20db-4a53-83e8-caf0225145d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.485711 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.485687 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:53:03.486760 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.486740 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:53:03.553104 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.553064 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:03.705857 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:03.705831 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7"] Jul 06 23:53:03.708824 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:53:03.708795 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24e7fdb_20db_4a53_83e8_caf0225145d1.slice/crio-3220c22e84a0266c48ca8d286c5c9151aa7bf3d2d30e1b9c61b6b5730bc35097 WatchSource:0}: Error finding container 3220c22e84a0266c48ca8d286c5c9151aa7bf3d2d30e1b9c61b6b5730bc35097: Status 404 returned error can't find the container with id 3220c22e84a0266c48ca8d286c5c9151aa7bf3d2d30e1b9c61b6b5730bc35097 Jul 06 23:53:04.492295 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:04.492261 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" event={"ID":"a24e7fdb-20db-4a53-83e8-caf0225145d1","Type":"ContainerStarted","Data":"3220c22e84a0266c48ca8d286c5c9151aa7bf3d2d30e1b9c61b6b5730bc35097"} Jul 06 23:53:06.325355 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.325321 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5"] Jul 06 23:53:06.332513 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.332489 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:06.335102 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.335080 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-w7whm\"" Jul 06 23:53:06.336847 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.336830 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 23:53:06.347389 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.347367 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5"] Jul 06 23:53:06.384680 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.384643 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm"] Jul 06 23:53:06.388632 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.388608 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:06.392319 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.392300 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 23:53:06.392436 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.392371 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fbhqd\"" Jul 06 23:53:06.402802 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.402780 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm"] Jul 06 23:53:06.502932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.502900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7682d489-0d6c-4b87-bdba-0cbcde4d6868-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hghqm\" (UID: \"7682d489-0d6c-4b87-bdba-0cbcde4d6868\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:06.503112 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.503025 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6bg8\" (UniqueName: \"kubernetes.io/projected/7682d489-0d6c-4b87-bdba-0cbcde4d6868-kube-api-access-p6bg8\") pod \"llmisvc-controller-manager-777c6bd7d5-hghqm\" (UID: \"7682d489-0d6c-4b87-bdba-0cbcde4d6868\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:06.503112 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.503064 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6frnx\" (UniqueName: \"kubernetes.io/projected/cc8e98a6-58cf-4d7e-a9ee-090c8939a6db-kube-api-access-6frnx\") pod \"kserve-controller-manager-d6f4b65b8-5w9s5\" (UID: \"cc8e98a6-58cf-4d7e-a9ee-090c8939a6db\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:06.503231 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.503118 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc8e98a6-58cf-4d7e-a9ee-090c8939a6db-cert\") pod \"kserve-controller-manager-d6f4b65b8-5w9s5\" (UID: \"cc8e98a6-58cf-4d7e-a9ee-090c8939a6db\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:06.604659 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.604576 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7682d489-0d6c-4b87-bdba-0cbcde4d6868-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hghqm\" (UID: \"7682d489-0d6c-4b87-bdba-0cbcde4d6868\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:06.604828 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.604661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6bg8\" (UniqueName: \"kubernetes.io/projected/7682d489-0d6c-4b87-bdba-0cbcde4d6868-kube-api-access-p6bg8\") pod \"llmisvc-controller-manager-777c6bd7d5-hghqm\" (UID: \"7682d489-0d6c-4b87-bdba-0cbcde4d6868\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:06.604828 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.604690 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6frnx\" (UniqueName: \"kubernetes.io/projected/cc8e98a6-58cf-4d7e-a9ee-090c8939a6db-kube-api-access-6frnx\") pod \"kserve-controller-manager-d6f4b65b8-5w9s5\" (UID: \"cc8e98a6-58cf-4d7e-a9ee-090c8939a6db\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:06.604828 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.604738 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc8e98a6-58cf-4d7e-a9ee-090c8939a6db-cert\") pod \"kserve-controller-manager-d6f4b65b8-5w9s5\" (UID: \"cc8e98a6-58cf-4d7e-a9ee-090c8939a6db\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:06.604828 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:53:06.604738 2583 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 06 23:53:06.605042 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:53:06.604886 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7682d489-0d6c-4b87-bdba-0cbcde4d6868-cert podName:7682d489-0d6c-4b87-bdba-0cbcde4d6868 nodeName:}" failed. No retries permitted until 2026-07-06 23:53:07.104861455 +0000 UTC m=+297.367533648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7682d489-0d6c-4b87-bdba-0cbcde4d6868-cert") pod "llmisvc-controller-manager-777c6bd7d5-hghqm" (UID: "7682d489-0d6c-4b87-bdba-0cbcde4d6868") : secret "llmisvc-webhook-server-cert" not found Jul 06 23:53:06.605106 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:53:06.605068 2583 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 06 23:53:06.605159 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:53:06.605131 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc8e98a6-58cf-4d7e-a9ee-090c8939a6db-cert podName:cc8e98a6-58cf-4d7e-a9ee-090c8939a6db nodeName:}" failed. No retries permitted until 2026-07-06 23:53:07.105113851 +0000 UTC m=+297.367786029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cc8e98a6-58cf-4d7e-a9ee-090c8939a6db-cert") pod "kserve-controller-manager-d6f4b65b8-5w9s5" (UID: "cc8e98a6-58cf-4d7e-a9ee-090c8939a6db") : secret "kserve-webhook-server-cert" not found Jul 06 23:53:06.691421 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.691393 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6frnx\" (UniqueName: \"kubernetes.io/projected/cc8e98a6-58cf-4d7e-a9ee-090c8939a6db-kube-api-access-6frnx\") pod \"kserve-controller-manager-d6f4b65b8-5w9s5\" (UID: \"cc8e98a6-58cf-4d7e-a9ee-090c8939a6db\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:06.692867 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.692842 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6bg8\" (UniqueName: \"kubernetes.io/projected/7682d489-0d6c-4b87-bdba-0cbcde4d6868-kube-api-access-p6bg8\") pod \"llmisvc-controller-manager-777c6bd7d5-hghqm\" (UID: \"7682d489-0d6c-4b87-bdba-0cbcde4d6868\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:06.925438 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.925405 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:53:06.925538 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.925493 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:53:06.925538 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:06.925528 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:53:07.109791 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.109749 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7682d489-0d6c-4b87-bdba-0cbcde4d6868-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hghqm\" (UID: \"7682d489-0d6c-4b87-bdba-0cbcde4d6868\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:07.109982 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.109883 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc8e98a6-58cf-4d7e-a9ee-090c8939a6db-cert\") pod \"kserve-controller-manager-d6f4b65b8-5w9s5\" (UID: \"cc8e98a6-58cf-4d7e-a9ee-090c8939a6db\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:07.112208 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.112176 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc8e98a6-58cf-4d7e-a9ee-090c8939a6db-cert\") pod \"kserve-controller-manager-d6f4b65b8-5w9s5\" (UID: \"cc8e98a6-58cf-4d7e-a9ee-090c8939a6db\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:07.112323 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.112229 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7682d489-0d6c-4b87-bdba-0cbcde4d6868-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-hghqm\" (UID: \"7682d489-0d6c-4b87-bdba-0cbcde4d6868\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:07.244910 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.244827 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:07.300283 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.300250 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:07.408079 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.408052 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5"] Jul 06 23:53:07.415114 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:53:07.413700 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc8e98a6_58cf_4d7e_a9ee_090c8939a6db.slice/crio-b47f145eb54e478526d11bcd01eb65884735282aa430593975c15efd2239ea33 WatchSource:0}: Error finding container b47f145eb54e478526d11bcd01eb65884735282aa430593975c15efd2239ea33: Status 404 returned error can't find the container with id b47f145eb54e478526d11bcd01eb65884735282aa430593975c15efd2239ea33 Jul 06 23:53:07.458823 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.458793 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm"] Jul 06 23:53:07.461442 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:53:07.461417 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7682d489_0d6c_4b87_bdba_0cbcde4d6868.slice/crio-06be305a62dd1ad015c2ccaaa8f5b0e245fe01b7b7853ca2b9fc118b70290284 WatchSource:0}: Error finding container 06be305a62dd1ad015c2ccaaa8f5b0e245fe01b7b7853ca2b9fc118b70290284: Status 404 returned error can't find the container with id 06be305a62dd1ad015c2ccaaa8f5b0e245fe01b7b7853ca2b9fc118b70290284 Jul 06 23:53:07.510109 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.510073 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" event={"ID":"cc8e98a6-58cf-4d7e-a9ee-090c8939a6db","Type":"ContainerStarted","Data":"b47f145eb54e478526d11bcd01eb65884735282aa430593975c15efd2239ea33"} Jul 06 23:53:07.511579 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.511558 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" event={"ID":"a24e7fdb-20db-4a53-83e8-caf0225145d1","Type":"ContainerStarted","Data":"37525dbe584c88c367af7b96ee49c405c4a9fb955e3f799a13ca58303ec8e2e1"} Jul 06 23:53:07.512679 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.512661 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" event={"ID":"7682d489-0d6c-4b87-bdba-0cbcde4d6868","Type":"ContainerStarted","Data":"06be305a62dd1ad015c2ccaaa8f5b0e245fe01b7b7853ca2b9fc118b70290284"} Jul 06 23:53:07.540076 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.540033 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" podStartSLOduration=1.325901353 podStartE2EDuration="4.540017948s" totalImagesPullingTime="3.214116595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:07.538943237 +0000 UTC m=+297.801615434" watchObservedRunningTime="2026-07-06 23:53:07.540017948 +0000 UTC m=+297.802690143" Jul 06 23:53:07.554010 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.553982 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:07.558346 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:07.558327 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:08.520654 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:08.520580 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:08.522105 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:08.522021 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t24l7" Jul 06 23:53:08.709939 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:08.709909 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nggsp"] Jul 06 23:53:08.710433 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:08.710188 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" podUID="3ba485b2-2db2-4647-ade3-a2c54ae4999e" containerName="discovery" containerID="cri-o://3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea" gracePeriod=30 Jul 06 23:53:09.006736 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.006686 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:53:09.128572 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.128493 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-token\") pod \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " Jul 06 23:53:09.128572 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.128535 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba485b2-2db2-4647-ade3-a2c54ae4999e-local-certs\") pod \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " Jul 06 23:53:09.128800 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.128602 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-ca-configmap\") pod \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " Jul 06 23:53:09.128800 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.128638 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82sgg\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-kube-api-access-82sgg\") pod \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " Jul 06 23:53:09.128800 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.128680 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-cacerts\") pod \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " Jul 06 23:53:09.128800 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.128708 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-dns-cert\") pod \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " Jul 06 23:53:09.128800 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.128779 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-kubeconfig\") pod \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\" (UID: \"3ba485b2-2db2-4647-ade3-a2c54ae4999e\") " Jul 06 23:53:09.129064 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.129004 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3ba485b2-2db2-4647-ade3-a2c54ae4999e" (UID: "3ba485b2-2db2-4647-ade3-a2c54ae4999e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:53:09.129148 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.129123 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-ca-configmap\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:53:09.131749 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.131717 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3ba485b2-2db2-4647-ade3-a2c54ae4999e" (UID: "3ba485b2-2db2-4647-ade3-a2c54ae4999e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:53:09.132180 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.132155 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba485b2-2db2-4647-ade3-a2c54ae4999e-local-certs" (OuterVolumeSpecName: "local-certs") pod "3ba485b2-2db2-4647-ade3-a2c54ae4999e" (UID: "3ba485b2-2db2-4647-ade3-a2c54ae4999e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:53:09.132295 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.132204 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-cacerts" (OuterVolumeSpecName: "cacerts") pod "3ba485b2-2db2-4647-ade3-a2c54ae4999e" (UID: "3ba485b2-2db2-4647-ade3-a2c54ae4999e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:53:09.132295 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.132245 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-kube-api-access-82sgg" (OuterVolumeSpecName: "kube-api-access-82sgg") pod "3ba485b2-2db2-4647-ade3-a2c54ae4999e" (UID: "3ba485b2-2db2-4647-ade3-a2c54ae4999e"). InnerVolumeSpecName "kube-api-access-82sgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:53:09.132404 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.132325 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3ba485b2-2db2-4647-ade3-a2c54ae4999e" (UID: "3ba485b2-2db2-4647-ade3-a2c54ae4999e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:53:09.133142 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.133113 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-token" (OuterVolumeSpecName: "istio-token") pod "3ba485b2-2db2-4647-ade3-a2c54ae4999e" (UID: "3ba485b2-2db2-4647-ade3-a2c54ae4999e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:53:09.230678 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.230633 2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-cacerts\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:53:09.230678 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.230673 2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-csr-dns-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:53:09.230919 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.230688 2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-kubeconfig\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:53:09.230919 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.230706 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:53:09.230919 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.230719 2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ba485b2-2db2-4647-ade3-a2c54ae4999e-local-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:53:09.230919 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.230732 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82sgg\" (UniqueName: \"kubernetes.io/projected/3ba485b2-2db2-4647-ade3-a2c54ae4999e-kube-api-access-82sgg\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:53:09.528385 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.528348 2583 generic.go:356] "Generic (PLEG): container finished" podID="3ba485b2-2db2-4647-ade3-a2c54ae4999e" containerID="3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea" exitCode=0 Jul 06 23:53:09.528866 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.528416 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" Jul 06 23:53:09.528866 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.528435 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" event={"ID":"3ba485b2-2db2-4647-ade3-a2c54ae4999e","Type":"ContainerDied","Data":"3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea"} Jul 06 23:53:09.528866 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.528513 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nggsp" event={"ID":"3ba485b2-2db2-4647-ade3-a2c54ae4999e","Type":"ContainerDied","Data":"200419e71239bb009f9fb625bc1593ab26987bc02fce20a6f966a45bd2090a4d"} Jul 06 23:53:09.528866 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.528541 2583 scope.go:117] "RemoveContainer" containerID="3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea" Jul 06 23:53:09.565420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.565387 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nggsp"] Jul 06 23:53:09.574272 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.574245 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nggsp"] Jul 06 23:53:09.700146 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.700125 2583 scope.go:117] "RemoveContainer" containerID="3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea" Jul 06 23:53:09.700521 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:53:09.700493 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea\": container with ID starting with 3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea not found: ID does not exist" containerID="3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea" Jul 06 23:53:09.700606 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:09.700534 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea"} err="failed to get container status \"3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea\": rpc error: code = NotFound desc = could not find container \"3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea\": container with ID starting with 3d2eb255a20fb202b73de85e94b76fb7e039fcad4b02c5874b449e8bb853a0ea not found: ID does not exist" Jul 06 23:53:10.183948 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:10.183781 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ba485b2-2db2-4647-ade3-a2c54ae4999e" path="/var/lib/kubelet/pods/3ba485b2-2db2-4647-ade3-a2c54ae4999e/volumes" Jul 06 23:53:10.365772 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:10.365749 2583 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 23:53:10.535462 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:10.535419 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" event={"ID":"cc8e98a6-58cf-4d7e-a9ee-090c8939a6db","Type":"ContainerStarted","Data":"fb65a86e600425c8f60a30bdaa5622694fefab81a3e4f77bca3b62ab6b26e73c"} Jul 06 23:53:10.535947 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:10.535623 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:10.537037 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:10.537009 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" event={"ID":"7682d489-0d6c-4b87-bdba-0cbcde4d6868","Type":"ContainerStarted","Data":"b8f9d09790afc307adbd6eaf9a1803ef0ca1320c9220bba325f23821b366330e"} Jul 06 23:53:10.537328 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:10.537280 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:10.575303 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:10.575259 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" podStartSLOduration=1.6623008430000001 podStartE2EDuration="4.575242326s" totalImagesPullingTime="2.912941483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:10.573120632 +0000 UTC m=+300.835792851" watchObservedRunningTime="2026-07-06 23:53:10.575242326 +0000 UTC m=+300.837914523" Jul 06 23:53:10.609505 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:10.609386 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" podStartSLOduration=1.7396752659999999 podStartE2EDuration="4.609370024s" totalImagesPullingTime="2.869694758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:53:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:10.608189513 +0000 UTC m=+300.870861708" watchObservedRunningTime="2026-07-06 23:53:10.609370024 +0000 UTC m=+300.872042219" Jul 06 23:53:13.487921 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:13.487885 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-tp54k" Jul 06 23:53:13.490105 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:13.490083 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-b456b" Jul 06 23:53:41.544419 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:41.544386 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-hghqm" Jul 06 23:53:41.547389 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:41.547368 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-5w9s5" Jul 06 23:53:55.740539 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.740503 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kbcc4"] Jul 06 23:53:55.741003 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.740896 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba485b2-2db2-4647-ade3-a2c54ae4999e" containerName="discovery" Jul 06 23:53:55.741003 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.740908 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba485b2-2db2-4647-ade3-a2c54ae4999e" containerName="discovery" Jul 06 23:53:55.741003 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.740978 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ba485b2-2db2-4647-ade3-a2c54ae4999e" containerName="discovery" Jul 06 23:53:55.747844 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.747813 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" Jul 06 23:53:55.749883 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.749854 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 23:53:55.749883 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.749879 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-b5bql\"" Jul 06 23:53:55.750402 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.750388 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 23:53:55.754379 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.754349 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kbcc4"] Jul 06 23:53:55.828817 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.828786 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nqrb\" (UniqueName: \"kubernetes.io/projected/4ca74764-ed5c-4743-910a-5d95edc8f399-kube-api-access-4nqrb\") pod \"kuadrant-operator-catalog-kbcc4\" (UID: \"4ca74764-ed5c-4743-910a-5d95edc8f399\") " pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" Jul 06 23:53:55.930166 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.930126 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nqrb\" (UniqueName: \"kubernetes.io/projected/4ca74764-ed5c-4743-910a-5d95edc8f399-kube-api-access-4nqrb\") pod \"kuadrant-operator-catalog-kbcc4\" (UID: \"4ca74764-ed5c-4743-910a-5d95edc8f399\") " pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" Jul 06 23:53:55.937735 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:55.937703 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nqrb\" (UniqueName: \"kubernetes.io/projected/4ca74764-ed5c-4743-910a-5d95edc8f399-kube-api-access-4nqrb\") pod \"kuadrant-operator-catalog-kbcc4\" (UID: \"4ca74764-ed5c-4743-910a-5d95edc8f399\") " pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" Jul 06 23:53:56.052050 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.052017 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kbcc4"] Jul 06 23:53:56.052317 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.052304 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" Jul 06 23:53:56.190402 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.190373 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:53:56.191770 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.191749 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kbcc4"] Jul 06 23:53:56.262574 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.262539 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9ppbd"] Jul 06 23:53:56.265319 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.265302 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" Jul 06 23:53:56.272583 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.272553 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9ppbd"] Jul 06 23:53:56.433845 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.433738 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7l9k\" (UniqueName: \"kubernetes.io/projected/1fd4387f-189a-4d28-8d78-3bb6f43b8381-kube-api-access-j7l9k\") pod \"kuadrant-operator-catalog-9ppbd\" (UID: \"1fd4387f-189a-4d28-8d78-3bb6f43b8381\") " pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" Jul 06 23:53:56.534775 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.534745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7l9k\" (UniqueName: \"kubernetes.io/projected/1fd4387f-189a-4d28-8d78-3bb6f43b8381-kube-api-access-j7l9k\") pod \"kuadrant-operator-catalog-9ppbd\" (UID: \"1fd4387f-189a-4d28-8d78-3bb6f43b8381\") " pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" Jul 06 23:53:56.542264 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.542234 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7l9k\" (UniqueName: \"kubernetes.io/projected/1fd4387f-189a-4d28-8d78-3bb6f43b8381-kube-api-access-j7l9k\") pod \"kuadrant-operator-catalog-9ppbd\" (UID: \"1fd4387f-189a-4d28-8d78-3bb6f43b8381\") " pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" Jul 06 23:53:56.575968 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.575937 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" Jul 06 23:53:56.709365 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.709283 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-9ppbd"] Jul 06 23:53:56.711543 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:53:56.711517 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd4387f_189a_4d28_8d78_3bb6f43b8381.slice/crio-1506631c29904774599aa3546c9069fe1b247f52d44c923f861ab116c9034f38 WatchSource:0}: Error finding container 1506631c29904774599aa3546c9069fe1b247f52d44c923f861ab116c9034f38: Status 404 returned error can't find the container with id 1506631c29904774599aa3546c9069fe1b247f52d44c923f861ab116c9034f38 Jul 06 23:53:56.769520 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.769440 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" event={"ID":"1fd4387f-189a-4d28-8d78-3bb6f43b8381","Type":"ContainerStarted","Data":"1506631c29904774599aa3546c9069fe1b247f52d44c923f861ab116c9034f38"} Jul 06 23:53:56.771722 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:56.771678 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" event={"ID":"4ca74764-ed5c-4743-910a-5d95edc8f399","Type":"ContainerStarted","Data":"2d9a09392e70baef0e60fb75c86f4fda3262283ec9f36514c89742b165aaf251"} Jul 06 23:53:58.784384 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:58.784341 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" event={"ID":"1fd4387f-189a-4d28-8d78-3bb6f43b8381","Type":"ContainerStarted","Data":"87c3182404302ae3c205235a0c6d56694817fd9c2ded6f74c946aeaa72168af2"} Jul 06 23:53:58.785953 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:58.785931 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" event={"ID":"4ca74764-ed5c-4743-910a-5d95edc8f399","Type":"ContainerStarted","Data":"d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b"} Jul 06 23:53:58.786028 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:58.785974 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" podUID="4ca74764-ed5c-4743-910a-5d95edc8f399" containerName="registry-server" containerID="cri-o://d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b" gracePeriod=2 Jul 06 23:53:58.800245 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:58.800202 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" podStartSLOduration=1.3593742 podStartE2EDuration="2.800186598s" totalImagesPullingTime="1.440812398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:58.799090215 +0000 UTC m=+349.061762432" watchObservedRunningTime="2026-07-06 23:53:58.800186598 +0000 UTC m=+349.062858829" Jul 06 23:53:58.818002 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:58.817953 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" podStartSLOduration=1.855181213 podStartE2EDuration="3.817936139s" totalImagesPullingTime="1.962754926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:53:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:53:58.815958407 +0000 UTC m=+349.078630604" watchObservedRunningTime="2026-07-06 23:53:58.817936139 +0000 UTC m=+349.080608334" Jul 06 23:53:59.039169 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.039111 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" Jul 06 23:53:59.159799 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.159766 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nqrb\" (UniqueName: \"kubernetes.io/projected/4ca74764-ed5c-4743-910a-5d95edc8f399-kube-api-access-4nqrb\") pod \"4ca74764-ed5c-4743-910a-5d95edc8f399\" (UID: \"4ca74764-ed5c-4743-910a-5d95edc8f399\") " Jul 06 23:53:59.161936 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.161905 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca74764-ed5c-4743-910a-5d95edc8f399-kube-api-access-4nqrb" (OuterVolumeSpecName: "kube-api-access-4nqrb") pod "4ca74764-ed5c-4743-910a-5d95edc8f399" (UID: "4ca74764-ed5c-4743-910a-5d95edc8f399"). InnerVolumeSpecName "kube-api-access-4nqrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:53:59.260992 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.260959 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nqrb\" (UniqueName: \"kubernetes.io/projected/4ca74764-ed5c-4743-910a-5d95edc8f399-kube-api-access-4nqrb\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:53:59.798712 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.798673 2583 generic.go:356] "Generic (PLEG): container finished" podID="4ca74764-ed5c-4743-910a-5d95edc8f399" containerID="d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b" exitCode=0 Jul 06 23:53:59.799194 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.798717 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" Jul 06 23:53:59.799194 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.798744 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" event={"ID":"4ca74764-ed5c-4743-910a-5d95edc8f399","Type":"ContainerDied","Data":"d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b"} Jul 06 23:53:59.799194 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.798821 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-kbcc4" event={"ID":"4ca74764-ed5c-4743-910a-5d95edc8f399","Type":"ContainerDied","Data":"2d9a09392e70baef0e60fb75c86f4fda3262283ec9f36514c89742b165aaf251"} Jul 06 23:53:59.799194 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.798841 2583 scope.go:117] "RemoveContainer" containerID="d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b" Jul 06 23:53:59.808158 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.808142 2583 scope.go:117] "RemoveContainer" containerID="d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b" Jul 06 23:53:59.808424 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:53:59.808406 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b\": container with ID starting with d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b not found: ID does not exist" containerID="d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b" Jul 06 23:53:59.808511 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.808435 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b"} err="failed to get container status \"d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b\": rpc error: code = NotFound desc = could not find container \"d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b\": container with ID starting with d4d56bf65d793f0306363e13eb30b598f6c587f3e3811503325082bffe01d58b not found: ID does not exist" Jul 06 23:53:59.818138 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.818114 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kbcc4"] Jul 06 23:53:59.821764 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:53:59.821746 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-kbcc4"] Jul 06 23:54:00.181236 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:00.181154 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca74764-ed5c-4743-910a-5d95edc8f399" path="/var/lib/kubelet/pods/4ca74764-ed5c-4743-910a-5d95edc8f399/volumes" Jul 06 23:54:06.576192 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:06.576087 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" Jul 06 23:54:06.576192 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:06.576180 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" Jul 06 23:54:06.600233 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:06.600210 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" Jul 06 23:54:06.855310 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:06.855236 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-9ppbd" Jul 06 23:54:11.496770 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.496734 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg"] Jul 06 23:54:11.497172 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.497091 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca74764-ed5c-4743-910a-5d95edc8f399" containerName="registry-server" Jul 06 23:54:11.497172 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.497103 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca74764-ed5c-4743-910a-5d95edc8f399" containerName="registry-server" Jul 06 23:54:11.497252 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.497182 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca74764-ed5c-4743-910a-5d95edc8f399" containerName="registry-server" Jul 06 23:54:11.500030 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.500013 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.502071 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.502053 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cxzj5\"" Jul 06 23:54:11.507186 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.506909 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg"] Jul 06 23:54:11.667500 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.667441 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.667500 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.667503 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.667735 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.667528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msrng\" (UniqueName: \"kubernetes.io/projected/d41a7224-0d15-4901-8657-b622da3d11cc-kube-api-access-msrng\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.768615 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.768521 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.768615 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.768568 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.768615 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.768588 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msrng\" (UniqueName: \"kubernetes.io/projected/d41a7224-0d15-4901-8657-b622da3d11cc-kube-api-access-msrng\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.768943 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.768924 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.769051 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.769032 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.776772 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.776750 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msrng\" (UniqueName: \"kubernetes.io/projected/d41a7224-0d15-4901-8657-b622da3d11cc-kube-api-access-msrng\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.809609 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.809575 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:11.941999 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:11.941977 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg"] Jul 06 23:54:11.943752 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:11.943725 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41a7224_0d15_4901_8657_b622da3d11cc.slice/crio-0c3520dff3d46afa8815562c3956ef3196c2bc341739d5b09ef1e4f5ce42ccd9 WatchSource:0}: Error finding container 0c3520dff3d46afa8815562c3956ef3196c2bc341739d5b09ef1e4f5ce42ccd9: Status 404 returned error can't find the container with id 0c3520dff3d46afa8815562c3956ef3196c2bc341739d5b09ef1e4f5ce42ccd9 Jul 06 23:54:12.100052 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.100017 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v"] Jul 06 23:54:12.102605 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.102589 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.110618 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.110594 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v"] Jul 06 23:54:12.272383 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.272349 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.272600 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.272401 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnhr4\" (UniqueName: \"kubernetes.io/projected/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-kube-api-access-xnhr4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.272600 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.272432 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.373741 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.373649 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.373893 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.373774 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.373893 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.373833 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnhr4\" (UniqueName: \"kubernetes.io/projected/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-kube-api-access-xnhr4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.374103 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.374082 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.374195 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.374132 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.381775 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.381749 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnhr4\" (UniqueName: \"kubernetes.io/projected/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-kube-api-access-xnhr4\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.429651 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.429614 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:12.500763 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.500735 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh"] Jul 06 23:54:12.504063 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.504045 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.512017 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.511986 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh"] Jul 06 23:54:12.568541 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.568517 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v"] Jul 06 23:54:12.570177 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:12.570148 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedccff2e_a93c_4bb4_afcf_0b99dfb66e7f.slice/crio-7847309e2f879237283f22b94a02e7175bc942b751051a9410151da11d00e789 WatchSource:0}: Error finding container 7847309e2f879237283f22b94a02e7175bc942b751051a9410151da11d00e789: Status 404 returned error can't find the container with id 7847309e2f879237283f22b94a02e7175bc942b751051a9410151da11d00e789 Jul 06 23:54:12.575280 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.575262 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.575324 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.575309 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w2pv\" (UniqueName: \"kubernetes.io/projected/56f2771f-27ec-46f3-ae8c-67172913f5aa-kube-api-access-6w2pv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.575358 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.575348 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.675751 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.675720 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w2pv\" (UniqueName: \"kubernetes.io/projected/56f2771f-27ec-46f3-ae8c-67172913f5aa-kube-api-access-6w2pv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.675880 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.675793 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.675880 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.675859 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.676206 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.676184 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.676242 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.676208 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.683737 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.683718 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w2pv\" (UniqueName: \"kubernetes.io/projected/56f2771f-27ec-46f3-ae8c-67172913f5aa-kube-api-access-6w2pv\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.815539 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.815496 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:12.865178 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.865071 2583 generic.go:356] "Generic (PLEG): container finished" podID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerID="d95e45898d1f89567aea93de6d5fcf133b858ef7cc5986c6e65045add818c3bc" exitCode=0 Jul 06 23:54:12.867071 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.866764 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" event={"ID":"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f","Type":"ContainerDied","Data":"d95e45898d1f89567aea93de6d5fcf133b858ef7cc5986c6e65045add818c3bc"} Jul 06 23:54:12.867071 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.866845 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" event={"ID":"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f","Type":"ContainerStarted","Data":"7847309e2f879237283f22b94a02e7175bc942b751051a9410151da11d00e789"} Jul 06 23:54:12.868880 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.868849 2583 generic.go:356] "Generic (PLEG): container finished" podID="d41a7224-0d15-4901-8657-b622da3d11cc" containerID="d7b8ddf95e2693a4c821637f6ac70db6e11285ec920caff16e3e2d2b3d9974de" exitCode=0 Jul 06 23:54:12.869000 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.868989 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" event={"ID":"d41a7224-0d15-4901-8657-b622da3d11cc","Type":"ContainerDied","Data":"d7b8ddf95e2693a4c821637f6ac70db6e11285ec920caff16e3e2d2b3d9974de"} Jul 06 23:54:12.869130 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.869028 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" event={"ID":"d41a7224-0d15-4901-8657-b622da3d11cc","Type":"ContainerStarted","Data":"0c3520dff3d46afa8815562c3956ef3196c2bc341739d5b09ef1e4f5ce42ccd9"} Jul 06 23:54:12.908513 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.908483 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477"] Jul 06 23:54:12.911369 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.911352 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:12.920156 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.919951 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477"] Jul 06 23:54:12.958391 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.958364 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh"] Jul 06 23:54:12.959147 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:12.959121 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56f2771f_27ec_46f3_ae8c_67172913f5aa.slice/crio-ea33f58ab8af26cee3ffa2abf7a7d1e969997ff75137abfa8a5b171aea12183d WatchSource:0}: Error finding container ea33f58ab8af26cee3ffa2abf7a7d1e969997ff75137abfa8a5b171aea12183d: Status 404 returned error can't find the container with id ea33f58ab8af26cee3ffa2abf7a7d1e969997ff75137abfa8a5b171aea12183d Jul 06 23:54:12.978609 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.978586 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:12.978745 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.978652 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:12.978806 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:12.978747 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-948qm\" (UniqueName: \"kubernetes.io/projected/d0bd98e0-34ef-47e3-8375-1a13e57a7789-kube-api-access-948qm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:13.079271 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.079236 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-948qm\" (UniqueName: \"kubernetes.io/projected/d0bd98e0-34ef-47e3-8375-1a13e57a7789-kube-api-access-948qm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:13.079395 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.079297 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:13.079395 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.079341 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:13.079668 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.079651 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:13.079721 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.079670 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:13.087544 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.087523 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-948qm\" (UniqueName: \"kubernetes.io/projected/d0bd98e0-34ef-47e3-8375-1a13e57a7789-kube-api-access-948qm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:13.223499 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.223385 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:13.359901 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.359773 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477"] Jul 06 23:54:13.362000 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:13.361976 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0bd98e0_34ef_47e3_8375_1a13e57a7789.slice/crio-12a0b117641c0fe287f1eea39f775246c45a24669b309d3c482131593cea5602 WatchSource:0}: Error finding container 12a0b117641c0fe287f1eea39f775246c45a24669b309d3c482131593cea5602: Status 404 returned error can't find the container with id 12a0b117641c0fe287f1eea39f775246c45a24669b309d3c482131593cea5602 Jul 06 23:54:13.876325 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.876293 2583 generic.go:356] "Generic (PLEG): container finished" podID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerID="d01d85e4226e59f2d5a909feea2ae87fbabf34432e170620bab2a1929dc66927" exitCode=0 Jul 06 23:54:13.876843 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.876380 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" event={"ID":"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f","Type":"ContainerDied","Data":"d01d85e4226e59f2d5a909feea2ae87fbabf34432e170620bab2a1929dc66927"} Jul 06 23:54:13.878031 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.878014 2583 generic.go:356] "Generic (PLEG): container finished" podID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerID="27b8df781640c8145159ce8c0bd50e826ccc6746fed78d7597cbfc7633f27c1c" exitCode=0 Jul 06 23:54:13.878163 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.878140 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" event={"ID":"d0bd98e0-34ef-47e3-8375-1a13e57a7789","Type":"ContainerDied","Data":"27b8df781640c8145159ce8c0bd50e826ccc6746fed78d7597cbfc7633f27c1c"} Jul 06 23:54:13.878279 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.878187 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" event={"ID":"d0bd98e0-34ef-47e3-8375-1a13e57a7789","Type":"ContainerStarted","Data":"12a0b117641c0fe287f1eea39f775246c45a24669b309d3c482131593cea5602"} Jul 06 23:54:13.879715 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.879693 2583 generic.go:356] "Generic (PLEG): container finished" podID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerID="6e5c5f1bec3a39d950b7a2c439e9c84aec7d863e80c049a8bde04dd806204e0a" exitCode=0 Jul 06 23:54:13.879849 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.879781 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" event={"ID":"56f2771f-27ec-46f3-ae8c-67172913f5aa","Type":"ContainerDied","Data":"6e5c5f1bec3a39d950b7a2c439e9c84aec7d863e80c049a8bde04dd806204e0a"} Jul 06 23:54:13.879849 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.879819 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" event={"ID":"56f2771f-27ec-46f3-ae8c-67172913f5aa","Type":"ContainerStarted","Data":"ea33f58ab8af26cee3ffa2abf7a7d1e969997ff75137abfa8a5b171aea12183d"} Jul 06 23:54:13.882237 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.881890 2583 generic.go:356] "Generic (PLEG): container finished" podID="d41a7224-0d15-4901-8657-b622da3d11cc" containerID="2c5d2da463ad18811ac266f641905679ede18bb900cd4d7e4d218272e3b6087d" exitCode=0 Jul 06 23:54:13.882237 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:13.881993 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" event={"ID":"d41a7224-0d15-4901-8657-b622da3d11cc","Type":"ContainerDied","Data":"2c5d2da463ad18811ac266f641905679ede18bb900cd4d7e4d218272e3b6087d"} Jul 06 23:54:14.888918 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:14.888834 2583 generic.go:356] "Generic (PLEG): container finished" podID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerID="39fd8bbef3c1ca8c86b079689d0d0e9a5afdd8092e0879a1dd5b4a1599613268" exitCode=0 Jul 06 23:54:14.889331 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:14.888924 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" event={"ID":"56f2771f-27ec-46f3-ae8c-67172913f5aa","Type":"ContainerDied","Data":"39fd8bbef3c1ca8c86b079689d0d0e9a5afdd8092e0879a1dd5b4a1599613268"} Jul 06 23:54:14.891167 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:14.891140 2583 generic.go:356] "Generic (PLEG): container finished" podID="d41a7224-0d15-4901-8657-b622da3d11cc" containerID="e7cc7fceef8c5c9515f21afd663057409e6acbcf40d3c0657bd0af50c06a5ebe" exitCode=0 Jul 06 23:54:14.891275 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:14.891213 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" event={"ID":"d41a7224-0d15-4901-8657-b622da3d11cc","Type":"ContainerDied","Data":"e7cc7fceef8c5c9515f21afd663057409e6acbcf40d3c0657bd0af50c06a5ebe"} Jul 06 23:54:14.893370 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:14.893347 2583 generic.go:356] "Generic (PLEG): container finished" podID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerID="fe032ecabee1d1771a089ab24eaae16a660ec04052d8f78e84abb040058a9f3c" exitCode=0 Jul 06 23:54:14.893453 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:14.893427 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" event={"ID":"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f","Type":"ContainerDied","Data":"fe032ecabee1d1771a089ab24eaae16a660ec04052d8f78e84abb040058a9f3c"} Jul 06 23:54:14.895176 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:14.895156 2583 generic.go:356] "Generic (PLEG): container finished" podID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerID="db03742f335a67ce2b5c481c9b3c83e25d4648394c08309f6576d4bcfe425614" exitCode=0 Jul 06 23:54:14.895268 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:14.895186 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" event={"ID":"d0bd98e0-34ef-47e3-8375-1a13e57a7789","Type":"ContainerDied","Data":"db03742f335a67ce2b5c481c9b3c83e25d4648394c08309f6576d4bcfe425614"} Jul 06 23:54:15.902343 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:15.902311 2583 generic.go:356] "Generic (PLEG): container finished" podID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerID="063d1d2b18caa3c9164426936083900f705708efc0c2f05e3614b1097c630729" exitCode=0 Jul 06 23:54:15.902783 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:15.902400 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" event={"ID":"d0bd98e0-34ef-47e3-8375-1a13e57a7789","Type":"ContainerDied","Data":"063d1d2b18caa3c9164426936083900f705708efc0c2f05e3614b1097c630729"} Jul 06 23:54:15.904331 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:15.904308 2583 generic.go:356] "Generic (PLEG): container finished" podID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerID="adcab4ff49f725af0cdad3d13c6b953dbf2352db44a55e4d507fe3f5da63c486" exitCode=0 Jul 06 23:54:15.904409 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:15.904389 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" event={"ID":"56f2771f-27ec-46f3-ae8c-67172913f5aa","Type":"ContainerDied","Data":"adcab4ff49f725af0cdad3d13c6b953dbf2352db44a55e4d507fe3f5da63c486"} Jul 06 23:54:16.037860 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.037835 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:16.072883 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.072856 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:16.102557 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.102526 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-util\") pod \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " Jul 06 23:54:16.102557 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.102567 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-util\") pod \"d41a7224-0d15-4901-8657-b622da3d11cc\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " Jul 06 23:54:16.102797 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.102587 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-bundle\") pod \"d41a7224-0d15-4901-8657-b622da3d11cc\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " Jul 06 23:54:16.102797 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.102618 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnhr4\" (UniqueName: \"kubernetes.io/projected/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-kube-api-access-xnhr4\") pod \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " Jul 06 23:54:16.102797 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.102638 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msrng\" (UniqueName: \"kubernetes.io/projected/d41a7224-0d15-4901-8657-b622da3d11cc-kube-api-access-msrng\") pod \"d41a7224-0d15-4901-8657-b622da3d11cc\" (UID: \"d41a7224-0d15-4901-8657-b622da3d11cc\") " Jul 06 23:54:16.102797 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.102707 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-bundle\") pod \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\" (UID: \"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f\") " Jul 06 23:54:16.103567 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.103432 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-bundle" (OuterVolumeSpecName: "bundle") pod "d41a7224-0d15-4901-8657-b622da3d11cc" (UID: "d41a7224-0d15-4901-8657-b622da3d11cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:16.103567 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.103525 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-bundle" (OuterVolumeSpecName: "bundle") pod "edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" (UID: "edccff2e-a93c-4bb4-afcf-0b99dfb66e7f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:16.104958 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.104934 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-kube-api-access-xnhr4" (OuterVolumeSpecName: "kube-api-access-xnhr4") pod "edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" (UID: "edccff2e-a93c-4bb4-afcf-0b99dfb66e7f"). InnerVolumeSpecName "kube-api-access-xnhr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:54:16.105176 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.105160 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d41a7224-0d15-4901-8657-b622da3d11cc-kube-api-access-msrng" (OuterVolumeSpecName: "kube-api-access-msrng") pod "d41a7224-0d15-4901-8657-b622da3d11cc" (UID: "d41a7224-0d15-4901-8657-b622da3d11cc"). InnerVolumeSpecName "kube-api-access-msrng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:54:16.108433 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.108404 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-util" (OuterVolumeSpecName: "util") pod "edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" (UID: "edccff2e-a93c-4bb4-afcf-0b99dfb66e7f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:16.108589 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.108568 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-util" (OuterVolumeSpecName: "util") pod "d41a7224-0d15-4901-8657-b622da3d11cc" (UID: "d41a7224-0d15-4901-8657-b622da3d11cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:16.204148 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.204123 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-util\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:16.204148 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.204147 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d41a7224-0d15-4901-8657-b622da3d11cc-bundle\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:16.204292 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.204157 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnhr4\" (UniqueName: \"kubernetes.io/projected/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-kube-api-access-xnhr4\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:16.204292 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.204166 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msrng\" (UniqueName: \"kubernetes.io/projected/d41a7224-0d15-4901-8657-b622da3d11cc-kube-api-access-msrng\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:16.204292 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.204180 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-bundle\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:16.204292 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.204189 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccff2e-a93c-4bb4-afcf-0b99dfb66e7f-util\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:16.912071 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.912038 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" Jul 06 23:54:16.912071 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.912057 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v" event={"ID":"edccff2e-a93c-4bb4-afcf-0b99dfb66e7f","Type":"ContainerDied","Data":"7847309e2f879237283f22b94a02e7175bc942b751051a9410151da11d00e789"} Jul 06 23:54:16.912594 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.912133 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7847309e2f879237283f22b94a02e7175bc942b751051a9410151da11d00e789" Jul 06 23:54:16.914217 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.914186 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" event={"ID":"d41a7224-0d15-4901-8657-b622da3d11cc","Type":"ContainerDied","Data":"0c3520dff3d46afa8815562c3956ef3196c2bc341739d5b09ef1e4f5ce42ccd9"} Jul 06 23:54:16.914327 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.914233 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c3520dff3d46afa8815562c3956ef3196c2bc341739d5b09ef1e4f5ce42ccd9" Jul 06 23:54:16.915259 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:16.914708 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg" Jul 06 23:54:17.060632 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.060570 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:17.090447 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.090415 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:17.110926 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.110899 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-util\") pod \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " Jul 06 23:54:17.111082 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.110960 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w2pv\" (UniqueName: \"kubernetes.io/projected/56f2771f-27ec-46f3-ae8c-67172913f5aa-kube-api-access-6w2pv\") pod \"56f2771f-27ec-46f3-ae8c-67172913f5aa\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " Jul 06 23:54:17.111082 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.110993 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-util\") pod \"56f2771f-27ec-46f3-ae8c-67172913f5aa\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " Jul 06 23:54:17.111082 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.111048 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-bundle\") pod \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " Jul 06 23:54:17.111249 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.111108 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-bundle\") pod \"56f2771f-27ec-46f3-ae8c-67172913f5aa\" (UID: \"56f2771f-27ec-46f3-ae8c-67172913f5aa\") " Jul 06 23:54:17.111249 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.111146 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-948qm\" (UniqueName: \"kubernetes.io/projected/d0bd98e0-34ef-47e3-8375-1a13e57a7789-kube-api-access-948qm\") pod \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\" (UID: \"d0bd98e0-34ef-47e3-8375-1a13e57a7789\") " Jul 06 23:54:17.112095 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.112063 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-bundle" (OuterVolumeSpecName: "bundle") pod "56f2771f-27ec-46f3-ae8c-67172913f5aa" (UID: "56f2771f-27ec-46f3-ae8c-67172913f5aa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:17.112228 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.112190 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-bundle" (OuterVolumeSpecName: "bundle") pod "d0bd98e0-34ef-47e3-8375-1a13e57a7789" (UID: "d0bd98e0-34ef-47e3-8375-1a13e57a7789"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:17.113271 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.113245 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56f2771f-27ec-46f3-ae8c-67172913f5aa-kube-api-access-6w2pv" (OuterVolumeSpecName: "kube-api-access-6w2pv") pod "56f2771f-27ec-46f3-ae8c-67172913f5aa" (UID: "56f2771f-27ec-46f3-ae8c-67172913f5aa"). InnerVolumeSpecName "kube-api-access-6w2pv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:54:17.113373 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.113308 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0bd98e0-34ef-47e3-8375-1a13e57a7789-kube-api-access-948qm" (OuterVolumeSpecName: "kube-api-access-948qm") pod "d0bd98e0-34ef-47e3-8375-1a13e57a7789" (UID: "d0bd98e0-34ef-47e3-8375-1a13e57a7789"). InnerVolumeSpecName "kube-api-access-948qm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:54:17.117256 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.117233 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-util" (OuterVolumeSpecName: "util") pod "d0bd98e0-34ef-47e3-8375-1a13e57a7789" (UID: "d0bd98e0-34ef-47e3-8375-1a13e57a7789"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:17.120005 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.119982 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-util" (OuterVolumeSpecName: "util") pod "56f2771f-27ec-46f3-ae8c-67172913f5aa" (UID: "56f2771f-27ec-46f3-ae8c-67172913f5aa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:17.212568 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.212534 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-util\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:17.212568 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.212562 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-bundle\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:17.212568 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.212570 2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/56f2771f-27ec-46f3-ae8c-67172913f5aa-bundle\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:17.212568 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.212579 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-948qm\" (UniqueName: \"kubernetes.io/projected/d0bd98e0-34ef-47e3-8375-1a13e57a7789-kube-api-access-948qm\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:17.212822 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.212588 2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d0bd98e0-34ef-47e3-8375-1a13e57a7789-util\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:17.212822 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.212596 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w2pv\" (UniqueName: \"kubernetes.io/projected/56f2771f-27ec-46f3-ae8c-67172913f5aa-kube-api-access-6w2pv\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:17.920804 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.920770 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" Jul 06 23:54:17.921245 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.920772 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477" event={"ID":"d0bd98e0-34ef-47e3-8375-1a13e57a7789","Type":"ContainerDied","Data":"12a0b117641c0fe287f1eea39f775246c45a24669b309d3c482131593cea5602"} Jul 06 23:54:17.921245 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.920905 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12a0b117641c0fe287f1eea39f775246c45a24669b309d3c482131593cea5602" Jul 06 23:54:17.922686 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.922662 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" event={"ID":"56f2771f-27ec-46f3-ae8c-67172913f5aa","Type":"ContainerDied","Data":"ea33f58ab8af26cee3ffa2abf7a7d1e969997ff75137abfa8a5b171aea12183d"} Jul 06 23:54:17.922767 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.922697 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh" Jul 06 23:54:17.922767 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:17.922705 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea33f58ab8af26cee3ffa2abf7a7d1e969997ff75137abfa8a5b171aea12183d" Jul 06 23:54:22.737774 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.737737 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l"] Jul 06 23:54:22.738164 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738109 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerName="util" Jul 06 23:54:22.738164 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738120 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerName="util" Jul 06 23:54:22.738164 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738132 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerName="extract" Jul 06 23:54:22.738164 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738138 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerName="extract" Jul 06 23:54:22.738164 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738146 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerName="util" Jul 06 23:54:22.738164 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738151 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerName="util" Jul 06 23:54:22.738164 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738158 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerName="util" Jul 06 23:54:22.738164 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738163 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerName="util" Jul 06 23:54:22.738164 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738169 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerName="pull" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738175 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerName="pull" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738186 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41a7224-0d15-4901-8657-b622da3d11cc" containerName="pull" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738192 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41a7224-0d15-4901-8657-b622da3d11cc" containerName="pull" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738197 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerName="extract" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738202 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerName="extract" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738208 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41a7224-0d15-4901-8657-b622da3d11cc" containerName="extract" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738213 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41a7224-0d15-4901-8657-b622da3d11cc" containerName="extract" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738220 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerName="pull" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738225 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerName="pull" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738235 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d41a7224-0d15-4901-8657-b622da3d11cc" containerName="util" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738240 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d41a7224-0d15-4901-8657-b622da3d11cc" containerName="util" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738250 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerName="pull" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738255 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerName="pull" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738262 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerName="extract" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738267 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerName="extract" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738322 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0bd98e0-34ef-47e3-8375-1a13e57a7789" containerName="extract" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738331 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="edccff2e-a93c-4bb4-afcf-0b99dfb66e7f" containerName="extract" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738338 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="d41a7224-0d15-4901-8657-b622da3d11cc" containerName="extract" Jul 06 23:54:22.738420 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.738343 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="56f2771f-27ec-46f3-ae8c-67172913f5aa" containerName="extract" Jul 06 23:54:22.742548 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.742528 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" Jul 06 23:54:22.744768 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.744739 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 23:54:22.744902 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.744786 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-6zqhw\"" Jul 06 23:54:22.750636 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.750612 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l"] Jul 06 23:54:22.763003 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.762977 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lvn6\" (UniqueName: \"kubernetes.io/projected/e3eb50fe-c234-4bbb-8b86-708e4b82430b-kube-api-access-9lvn6\") pod \"dns-operator-controller-manager-864dd78c4c-h542l\" (UID: \"e3eb50fe-c234-4bbb-8b86-708e4b82430b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" Jul 06 23:54:22.864256 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.864225 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lvn6\" (UniqueName: \"kubernetes.io/projected/e3eb50fe-c234-4bbb-8b86-708e4b82430b-kube-api-access-9lvn6\") pod \"dns-operator-controller-manager-864dd78c4c-h542l\" (UID: \"e3eb50fe-c234-4bbb-8b86-708e4b82430b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" Jul 06 23:54:22.875517 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:22.875494 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lvn6\" (UniqueName: \"kubernetes.io/projected/e3eb50fe-c234-4bbb-8b86-708e4b82430b-kube-api-access-9lvn6\") pod \"dns-operator-controller-manager-864dd78c4c-h542l\" (UID: \"e3eb50fe-c234-4bbb-8b86-708e4b82430b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" Jul 06 23:54:23.054150 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:23.054112 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" Jul 06 23:54:23.196253 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:23.196221 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l"] Jul 06 23:54:23.200141 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:23.200115 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3eb50fe_c234_4bbb_8b86_708e4b82430b.slice/crio-265087733dfb9da3d44355d1a969811db2e03f6c38af4e711cf83ac72d13c6a7 WatchSource:0}: Error finding container 265087733dfb9da3d44355d1a969811db2e03f6c38af4e711cf83ac72d13c6a7: Status 404 returned error can't find the container with id 265087733dfb9da3d44355d1a969811db2e03f6c38af4e711cf83ac72d13c6a7 Jul 06 23:54:23.957251 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:23.957195 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" event={"ID":"e3eb50fe-c234-4bbb-8b86-708e4b82430b","Type":"ContainerStarted","Data":"265087733dfb9da3d44355d1a969811db2e03f6c38af4e711cf83ac72d13c6a7"} Jul 06 23:54:24.964991 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:24.964965 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" event={"ID":"e3eb50fe-c234-4bbb-8b86-708e4b82430b","Type":"ContainerStarted","Data":"b79528778a896b2769b6622e0e56fc1c2d7eb020c225017337a37dd027a6dc43"} Jul 06 23:54:24.965380 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:24.965023 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" Jul 06 23:54:24.981372 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:24.981330 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" podStartSLOduration=1.294029791 podStartE2EDuration="2.981314824s" totalImagesPullingTime="1.687285033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:24.980328234 +0000 UTC m=+375.243000431" watchObservedRunningTime="2026-07-06 23:54:24.981314824 +0000 UTC m=+375.243987018" Jul 06 23:54:27.917736 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:27.917698 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp"] Jul 06 23:54:27.924602 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:27.924574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:27.931319 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:27.931298 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mh67n\"" Jul 06 23:54:27.942921 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:27.942893 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp"] Jul 06 23:54:28.010197 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:28.010159 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42b75373-b969-4e27-b33e-171a8ff5cb29-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" (UID: \"42b75373-b969-4e27-b33e-171a8ff5cb29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:28.010370 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:28.010217 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvb6n\" (UniqueName: \"kubernetes.io/projected/42b75373-b969-4e27-b33e-171a8ff5cb29-kube-api-access-vvb6n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" (UID: \"42b75373-b969-4e27-b33e-171a8ff5cb29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:28.111523 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:28.111467 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42b75373-b969-4e27-b33e-171a8ff5cb29-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" (UID: \"42b75373-b969-4e27-b33e-171a8ff5cb29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:28.111691 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:28.111558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvb6n\" (UniqueName: \"kubernetes.io/projected/42b75373-b969-4e27-b33e-171a8ff5cb29-kube-api-access-vvb6n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" (UID: \"42b75373-b969-4e27-b33e-171a8ff5cb29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:28.111843 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:28.111824 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42b75373-b969-4e27-b33e-171a8ff5cb29-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" (UID: \"42b75373-b969-4e27-b33e-171a8ff5cb29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:28.119015 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:28.118979 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvb6n\" (UniqueName: \"kubernetes.io/projected/42b75373-b969-4e27-b33e-171a8ff5cb29-kube-api-access-vvb6n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" (UID: \"42b75373-b969-4e27-b33e-171a8ff5cb29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:28.234557 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:28.234428 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:28.383127 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:28.383097 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp"] Jul 06 23:54:28.384723 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:28.384701 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b75373_b969_4e27_b33e_171a8ff5cb29.slice/crio-6d8ccb9914dd96c636c9f5dfe9a0cb7340bca6bdcc1c90ef2d1e0e0ad7745e0b WatchSource:0}: Error finding container 6d8ccb9914dd96c636c9f5dfe9a0cb7340bca6bdcc1c90ef2d1e0e0ad7745e0b: Status 404 returned error can't find the container with id 6d8ccb9914dd96c636c9f5dfe9a0cb7340bca6bdcc1c90ef2d1e0e0ad7745e0b Jul 06 23:54:28.987934 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:28.987892 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" event={"ID":"42b75373-b969-4e27-b33e-171a8ff5cb29","Type":"ContainerStarted","Data":"6d8ccb9914dd96c636c9f5dfe9a0cb7340bca6bdcc1c90ef2d1e0e0ad7745e0b"} Jul 06 23:54:31.668326 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:31.668287 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-npdkl"] Jul 06 23:54:31.672295 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:31.672275 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" Jul 06 23:54:31.674552 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:31.674511 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-fhxzr\"" Jul 06 23:54:31.681693 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:31.681261 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-npdkl"] Jul 06 23:54:31.747097 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:31.747057 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh78c\" (UniqueName: \"kubernetes.io/projected/a147ff07-4edc-434e-abfe-c0e8bcfa1452-kube-api-access-zh78c\") pod \"authorino-operator-75cc8d5cb7-npdkl\" (UID: \"a147ff07-4edc-434e-abfe-c0e8bcfa1452\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" Jul 06 23:54:31.848318 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:31.848279 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh78c\" (UniqueName: \"kubernetes.io/projected/a147ff07-4edc-434e-abfe-c0e8bcfa1452-kube-api-access-zh78c\") pod \"authorino-operator-75cc8d5cb7-npdkl\" (UID: \"a147ff07-4edc-434e-abfe-c0e8bcfa1452\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" Jul 06 23:54:31.857047 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:31.857013 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh78c\" (UniqueName: \"kubernetes.io/projected/a147ff07-4edc-434e-abfe-c0e8bcfa1452-kube-api-access-zh78c\") pod \"authorino-operator-75cc8d5cb7-npdkl\" (UID: \"a147ff07-4edc-434e-abfe-c0e8bcfa1452\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" Jul 06 23:54:31.987021 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:31.986929 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" Jul 06 23:54:32.936724 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:32.936699 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-npdkl"] Jul 06 23:54:32.938765 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:32.938741 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda147ff07_4edc_434e_abfe_c0e8bcfa1452.slice/crio-c3c97cb89428074195569adf81afa7cca4900f6d0c98043daa268fc5b455e49c WatchSource:0}: Error finding container c3c97cb89428074195569adf81afa7cca4900f6d0c98043daa268fc5b455e49c: Status 404 returned error can't find the container with id c3c97cb89428074195569adf81afa7cca4900f6d0c98043daa268fc5b455e49c Jul 06 23:54:33.011977 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:33.011937 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" event={"ID":"42b75373-b969-4e27-b33e-171a8ff5cb29","Type":"ContainerStarted","Data":"b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db"} Jul 06 23:54:33.012232 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:33.012097 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:33.013209 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:33.013187 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" event={"ID":"a147ff07-4edc-434e-abfe-c0e8bcfa1452","Type":"ContainerStarted","Data":"c3c97cb89428074195569adf81afa7cca4900f6d0c98043daa268fc5b455e49c"} Jul 06 23:54:33.033528 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:33.033456 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" podStartSLOduration=1.565707937 podStartE2EDuration="6.033438056s" totalImagesPullingTime="4.467730119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:33.03216834 +0000 UTC m=+383.294840537" watchObservedRunningTime="2026-07-06 23:54:33.033438056 +0000 UTC m=+383.296110251" Jul 06 23:54:35.973082 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:35.973051 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-h542l" Jul 06 23:54:35.974970 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:35.974945 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv"] Jul 06 23:54:35.978834 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:35.978811 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:35.980834 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:35.980811 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 23:54:35.980952 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:35.980815 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 23:54:35.980952 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:35.980820 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cxzj5\"" Jul 06 23:54:35.985602 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:35.985583 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv"] Jul 06 23:54:36.033396 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.033351 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" event={"ID":"a147ff07-4edc-434e-abfe-c0e8bcfa1452","Type":"ContainerStarted","Data":"cb3112b822d3fbe52cf7b32486a40169005e97c83a52e172b2712fd165b29982"} Jul 06 23:54:36.033575 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.033501 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" Jul 06 23:54:36.053447 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.053396 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" podStartSLOduration=2.878508417 podStartE2EDuration="5.053380421s" totalImagesPullingTime="2.174872004s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:36.051539935 +0000 UTC m=+386.314212129" watchObservedRunningTime="2026-07-06 23:54:36.053380421 +0000 UTC m=+386.316052616" Jul 06 23:54:36.091240 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.091203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/975e7a09-fc30-4d58-a83a-dc98a208b07e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-62tvv\" (UID: \"975e7a09-fc30-4d58-a83a-dc98a208b07e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.091403 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.091325 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/975e7a09-fc30-4d58-a83a-dc98a208b07e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-62tvv\" (UID: \"975e7a09-fc30-4d58-a83a-dc98a208b07e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.091403 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.091380 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbtkl\" (UniqueName: \"kubernetes.io/projected/975e7a09-fc30-4d58-a83a-dc98a208b07e-kube-api-access-sbtkl\") pod \"kuadrant-console-plugin-66b5774985-62tvv\" (UID: \"975e7a09-fc30-4d58-a83a-dc98a208b07e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.192089 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.192052 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/975e7a09-fc30-4d58-a83a-dc98a208b07e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-62tvv\" (UID: \"975e7a09-fc30-4d58-a83a-dc98a208b07e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.192232 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.192102 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/975e7a09-fc30-4d58-a83a-dc98a208b07e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-62tvv\" (UID: \"975e7a09-fc30-4d58-a83a-dc98a208b07e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.192232 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.192131 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbtkl\" (UniqueName: \"kubernetes.io/projected/975e7a09-fc30-4d58-a83a-dc98a208b07e-kube-api-access-sbtkl\") pod \"kuadrant-console-plugin-66b5774985-62tvv\" (UID: \"975e7a09-fc30-4d58-a83a-dc98a208b07e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.192793 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.192769 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/975e7a09-fc30-4d58-a83a-dc98a208b07e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-62tvv\" (UID: \"975e7a09-fc30-4d58-a83a-dc98a208b07e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.194439 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.194419 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/975e7a09-fc30-4d58-a83a-dc98a208b07e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-62tvv\" (UID: \"975e7a09-fc30-4d58-a83a-dc98a208b07e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.199613 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.199590 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbtkl\" (UniqueName: \"kubernetes.io/projected/975e7a09-fc30-4d58-a83a-dc98a208b07e-kube-api-access-sbtkl\") pod \"kuadrant-console-plugin-66b5774985-62tvv\" (UID: \"975e7a09-fc30-4d58-a83a-dc98a208b07e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.289557 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.289520 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" Jul 06 23:54:36.417591 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:36.417563 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv"] Jul 06 23:54:36.419384 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:36.419344 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod975e7a09_fc30_4d58_a83a_dc98a208b07e.slice/crio-7b4ac2848fb0a52e42756b1e471dd2d64ee906bb43280695417687ab5c17efa1 WatchSource:0}: Error finding container 7b4ac2848fb0a52e42756b1e471dd2d64ee906bb43280695417687ab5c17efa1: Status 404 returned error can't find the container with id 7b4ac2848fb0a52e42756b1e471dd2d64ee906bb43280695417687ab5c17efa1 Jul 06 23:54:37.042651 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:37.042608 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" event={"ID":"975e7a09-fc30-4d58-a83a-dc98a208b07e","Type":"ContainerStarted","Data":"7b4ac2848fb0a52e42756b1e471dd2d64ee906bb43280695417687ab5c17efa1"} Jul 06 23:54:44.022241 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:44.022205 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:45.830902 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.830862 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp"] Jul 06 23:54:45.831366 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.831099 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" containerName="manager" containerID="cri-o://b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db" gracePeriod=2 Jul 06 23:54:45.839941 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.839914 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp"] Jul 06 23:54:45.841936 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:54:45.841900 2583 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"42b75373-b969-4e27-b33e-171a8ff5cb29\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-psjtp\": pods \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:45.860329 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.860296 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6"] Jul 06 23:54:45.860840 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.860821 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" containerName="manager" Jul 06 23:54:45.860840 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.860841 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" containerName="manager" Jul 06 23:54:45.860988 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.860946 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" containerName="manager" Jul 06 23:54:45.875099 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.875076 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:54:45.878313 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:54:45.878272 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" is forbidden: User \"system:node:ip-10-0-137-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-188.ec2.internal' and this object" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:45.881534 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.881506 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6"] Jul 06 23:54:45.892732 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.892698 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644"] Jul 06 23:54:45.897742 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.897720 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" Jul 06 23:54:45.900206 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.900178 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-pp4gb\"" Jul 06 23:54:45.906486 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.906449 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644"] Jul 06 23:54:45.938398 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:54:45.938345 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" is forbidden: User \"system:node:ip-10-0-137-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-188.ec2.internal' and this object" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:45.991198 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.991146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dmpq\" (UniqueName: \"kubernetes.io/projected/13435547-2d5c-4324-9a31-95d18fb2f6ee-kube-api-access-5dmpq\") pod \"limitador-operator-controller-manager-859d7b78bd-s6644\" (UID: \"13435547-2d5c-4324-9a31-95d18fb2f6ee\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" Jul 06 23:54:45.991397 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.991204 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvtlg\" (UniqueName: \"kubernetes.io/projected/5fff1d56-a2bc-44bf-9370-410dde9bf254-kube-api-access-hvtlg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rn4c6\" (UID: \"5fff1d56-a2bc-44bf-9370-410dde9bf254\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:54:45.991397 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:45.991271 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fff1d56-a2bc-44bf-9370-410dde9bf254-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rn4c6\" (UID: \"5fff1d56-a2bc-44bf-9370-410dde9bf254\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:54:46.092839 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:46.092747 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dmpq\" (UniqueName: \"kubernetes.io/projected/13435547-2d5c-4324-9a31-95d18fb2f6ee-kube-api-access-5dmpq\") pod \"limitador-operator-controller-manager-859d7b78bd-s6644\" (UID: \"13435547-2d5c-4324-9a31-95d18fb2f6ee\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" Jul 06 23:54:46.092839 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:46.092798 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvtlg\" (UniqueName: \"kubernetes.io/projected/5fff1d56-a2bc-44bf-9370-410dde9bf254-kube-api-access-hvtlg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rn4c6\" (UID: \"5fff1d56-a2bc-44bf-9370-410dde9bf254\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:54:46.093064 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:46.092853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fff1d56-a2bc-44bf-9370-410dde9bf254-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rn4c6\" (UID: \"5fff1d56-a2bc-44bf-9370-410dde9bf254\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:54:46.093313 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:46.093286 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fff1d56-a2bc-44bf-9370-410dde9bf254-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rn4c6\" (UID: \"5fff1d56-a2bc-44bf-9370-410dde9bf254\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:54:46.101890 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:46.101863 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvtlg\" (UniqueName: \"kubernetes.io/projected/5fff1d56-a2bc-44bf-9370-410dde9bf254-kube-api-access-hvtlg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rn4c6\" (UID: \"5fff1d56-a2bc-44bf-9370-410dde9bf254\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:54:46.102184 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:46.102164 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dmpq\" (UniqueName: \"kubernetes.io/projected/13435547-2d5c-4324-9a31-95d18fb2f6ee-kube-api-access-5dmpq\") pod \"limitador-operator-controller-manager-859d7b78bd-s6644\" (UID: \"13435547-2d5c-4324-9a31-95d18fb2f6ee\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" Jul 06 23:54:46.233224 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:46.233184 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:54:46.241375 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:46.241339 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" Jul 06 23:54:47.045850 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:47.045819 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-npdkl" Jul 06 23:54:57.594622 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:57.594591 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644"] Jul 06 23:54:57.594934 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:57.594912 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:57.596840 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:54:57.596815 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" is forbidden: User \"system:node:ip-10-0-137-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-188.ec2.internal' and this object" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:57.600098 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:57.600078 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42b75373-b969-4e27-b33e-171a8ff5cb29-extensions-socket-volume\") pod \"42b75373-b969-4e27-b33e-171a8ff5cb29\" (UID: \"42b75373-b969-4e27-b33e-171a8ff5cb29\") " Jul 06 23:54:57.600183 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:57.600131 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvb6n\" (UniqueName: \"kubernetes.io/projected/42b75373-b969-4e27-b33e-171a8ff5cb29-kube-api-access-vvb6n\") pod \"42b75373-b969-4e27-b33e-171a8ff5cb29\" (UID: \"42b75373-b969-4e27-b33e-171a8ff5cb29\") " Jul 06 23:54:57.600533 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:57.600506 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b75373-b969-4e27-b33e-171a8ff5cb29-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "42b75373-b969-4e27-b33e-171a8ff5cb29" (UID: "42b75373-b969-4e27-b33e-171a8ff5cb29"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:54:57.602204 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:57.602179 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b75373-b969-4e27-b33e-171a8ff5cb29-kube-api-access-vvb6n" (OuterVolumeSpecName: "kube-api-access-vvb6n") pod "42b75373-b969-4e27-b33e-171a8ff5cb29" (UID: "42b75373-b969-4e27-b33e-171a8ff5cb29"). InnerVolumeSpecName "kube-api-access-vvb6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:54:57.603781 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:57.603755 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13435547_2d5c_4324_9a31_95d18fb2f6ee.slice/crio-2b41d68bb4b23d4d9ef7120153b2133402b5a245a082eb907bd1c34d9462f7b6 WatchSource:0}: Error finding container 2b41d68bb4b23d4d9ef7120153b2133402b5a245a082eb907bd1c34d9462f7b6: Status 404 returned error can't find the container with id 2b41d68bb4b23d4d9ef7120153b2133402b5a245a082eb907bd1c34d9462f7b6 Jul 06 23:54:57.625040 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:57.625024 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6"] Jul 06 23:54:57.627375 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:54:57.627353 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fff1d56_a2bc_44bf_9370_410dde9bf254.slice/crio-ad840ed456e8b8a982d217d189c49c39fc3811c84cf7fd04d1d146b0cee30937 WatchSource:0}: Error finding container ad840ed456e8b8a982d217d189c49c39fc3811c84cf7fd04d1d146b0cee30937: Status 404 returned error can't find the container with id ad840ed456e8b8a982d217d189c49c39fc3811c84cf7fd04d1d146b0cee30937 Jul 06 23:54:57.701672 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:57.701637 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/42b75373-b969-4e27-b33e-171a8ff5cb29-extensions-socket-volume\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:57.701672 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:57.701671 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvb6n\" (UniqueName: \"kubernetes.io/projected/42b75373-b969-4e27-b33e-171a8ff5cb29-kube-api-access-vvb6n\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:54:58.181589 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:54:58.181546 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" is forbidden: User \"system:node:ip-10-0-137-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-188.ec2.internal' and this object" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:58.181833 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.181803 2583 generic.go:356] "Generic (PLEG): container finished" podID="42b75373-b969-4e27-b33e-171a8ff5cb29" containerID="b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db" exitCode=0 Jul 06 23:54:58.181962 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.181949 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:58.186157 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.185923 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" path="/var/lib/kubelet/pods/42b75373-b969-4e27-b33e-171a8ff5cb29/volumes" Jul 06 23:54:58.186393 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.186371 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" event={"ID":"975e7a09-fc30-4d58-a83a-dc98a208b07e","Type":"ContainerStarted","Data":"9fd2c716335aceb4d01f1c14af8e84dd6dd037d64a2378c629252871c6a9faa0"} Jul 06 23:54:58.186491 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.186413 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:54:58.186491 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.186454 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" event={"ID":"5fff1d56-a2bc-44bf-9370-410dde9bf254","Type":"ContainerStarted","Data":"dd0818434c63b463b5a4ff8d3fc21a6339018833949cd51afa83289f5e27d30a"} Jul 06 23:54:58.186590 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.186494 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" event={"ID":"5fff1d56-a2bc-44bf-9370-410dde9bf254","Type":"ContainerStarted","Data":"ad840ed456e8b8a982d217d189c49c39fc3811c84cf7fd04d1d146b0cee30937"} Jul 06 23:54:58.186590 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.186518 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" event={"ID":"13435547-2d5c-4324-9a31-95d18fb2f6ee","Type":"ContainerStarted","Data":"2b41d68bb4b23d4d9ef7120153b2133402b5a245a082eb907bd1c34d9462f7b6"} Jul 06 23:54:58.186590 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.186548 2583 scope.go:117] "RemoveContainer" containerID="b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db" Jul 06 23:54:58.198628 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.198580 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-62tvv" podStartSLOduration=2.089556019 podStartE2EDuration="23.198560942s" totalImagesPullingTime="21.109004923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:58.195550366 +0000 UTC m=+408.458222561" watchObservedRunningTime="2026-07-06 23:54:58.198560942 +0000 UTC m=+408.461233139" Jul 06 23:54:58.216181 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:54:58.216131 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" is forbidden: User \"system:node:ip-10-0-137-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-188.ec2.internal' and this object" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:58.218505 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.218427 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" podStartSLOduration=13.218402636 podStartE2EDuration="13.218402636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:58.212918829 +0000 UTC m=+408.475591026" watchObservedRunningTime="2026-07-06 23:54:58.218402636 +0000 UTC m=+408.481074833" Jul 06 23:54:58.234621 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.234592 2583 scope.go:117] "RemoveContainer" containerID="b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db" Jul 06 23:54:58.238163 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:54:58.238124 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db\": container with ID starting with b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db not found: ID does not exist" containerID="b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db" Jul 06 23:54:58.238559 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:58.238175 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db"} err="failed to get container status \"b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db\": rpc error: code = NotFound desc = could not find container \"b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db\": container with ID starting with b61a1a1af5b59793c6684e4fa681b804df6247f9767ca91011089228a76669db not found: ID does not exist" Jul 06 23:54:59.194390 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:59.194280 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" event={"ID":"13435547-2d5c-4324-9a31-95d18fb2f6ee","Type":"ContainerStarted","Data":"ba0b2ba775f613c735492780318ec62b29a1d8e5257e55693e10b59fb649493c"} Jul 06 23:54:59.194847 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:59.194687 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" Jul 06 23:54:59.210315 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:54:59.210281 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" is forbidden: User \"system:node:ip-10-0-137-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-188.ec2.internal' and this object" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:54:59.211238 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:54:59.211201 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" podStartSLOduration=12.975485539 podStartE2EDuration="14.211188216s" totalImagesPullingTime="1.235702677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:54:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:54:59.208574732 +0000 UTC m=+409.471246939" watchObservedRunningTime="2026-07-06 23:54:59.211188216 +0000 UTC m=+409.473860411" Jul 06 23:55:00.095088 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:00.095049 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654cf8646-mttr5"] Jul 06 23:55:00.114376 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:55:00.114334 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" is forbidden: User \"system:node:ip-10-0-137-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-188.ec2.internal' and this object" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:55:00.180004 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:55:00.179968 2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-psjtp\" is forbidden: User \"system:node:ip-10-0-137-188.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-188.ec2.internal' and this object" podUID="42b75373-b969-4e27-b33e-171a8ff5cb29" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-psjtp" Jul 06 23:55:03.073289 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.073253 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6"] Jul 06 23:55:03.073729 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.073437 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" podUID="5fff1d56-a2bc-44bf-9370-410dde9bf254" containerName="manager" containerID="cri-o://dd0818434c63b463b5a4ff8d3fc21a6339018833949cd51afa83289f5e27d30a" gracePeriod=10 Jul 06 23:55:03.075370 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.075347 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:55:03.219008 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.218978 2583 generic.go:356] "Generic (PLEG): container finished" podID="5fff1d56-a2bc-44bf-9370-410dde9bf254" containerID="dd0818434c63b463b5a4ff8d3fc21a6339018833949cd51afa83289f5e27d30a" exitCode=0 Jul 06 23:55:03.219173 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.219036 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" event={"ID":"5fff1d56-a2bc-44bf-9370-410dde9bf254","Type":"ContainerDied","Data":"dd0818434c63b463b5a4ff8d3fc21a6339018833949cd51afa83289f5e27d30a"} Jul 06 23:55:03.320619 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.320595 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:55:03.352814 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.352746 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvtlg\" (UniqueName: \"kubernetes.io/projected/5fff1d56-a2bc-44bf-9370-410dde9bf254-kube-api-access-hvtlg\") pod \"5fff1d56-a2bc-44bf-9370-410dde9bf254\" (UID: \"5fff1d56-a2bc-44bf-9370-410dde9bf254\") " Jul 06 23:55:03.352814 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.352797 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fff1d56-a2bc-44bf-9370-410dde9bf254-extensions-socket-volume\") pod \"5fff1d56-a2bc-44bf-9370-410dde9bf254\" (UID: \"5fff1d56-a2bc-44bf-9370-410dde9bf254\") " Jul 06 23:55:03.353175 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.353154 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fff1d56-a2bc-44bf-9370-410dde9bf254-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5fff1d56-a2bc-44bf-9370-410dde9bf254" (UID: "5fff1d56-a2bc-44bf-9370-410dde9bf254"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 23:55:03.354809 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.354786 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fff1d56-a2bc-44bf-9370-410dde9bf254-kube-api-access-hvtlg" (OuterVolumeSpecName: "kube-api-access-hvtlg") pod "5fff1d56-a2bc-44bf-9370-410dde9bf254" (UID: "5fff1d56-a2bc-44bf-9370-410dde9bf254"). InnerVolumeSpecName "kube-api-access-hvtlg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:03.454156 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.454121 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvtlg\" (UniqueName: \"kubernetes.io/projected/5fff1d56-a2bc-44bf-9370-410dde9bf254-kube-api-access-hvtlg\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:03.454156 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:03.454148 2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fff1d56-a2bc-44bf-9370-410dde9bf254-extensions-socket-volume\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:04.226424 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:04.226390 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" event={"ID":"5fff1d56-a2bc-44bf-9370-410dde9bf254","Type":"ContainerDied","Data":"ad840ed456e8b8a982d217d189c49c39fc3811c84cf7fd04d1d146b0cee30937"} Jul 06 23:55:04.226878 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:04.226454 2583 scope.go:117] "RemoveContainer" containerID="dd0818434c63b463b5a4ff8d3fc21a6339018833949cd51afa83289f5e27d30a" Jul 06 23:55:04.226878 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:04.226401 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6" Jul 06 23:55:04.259529 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:04.259498 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6"] Jul 06 23:55:04.263467 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:04.263442 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rn4c6"] Jul 06 23:55:06.181849 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:06.181816 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fff1d56-a2bc-44bf-9370-410dde9bf254" path="/var/lib/kubelet/pods/5fff1d56-a2bc-44bf-9370-410dde9bf254/volumes" Jul 06 23:55:10.202091 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:10.202065 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-s6644" Jul 06 23:55:19.459141 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.459105 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr"] Jul 06 23:55:19.459676 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.459657 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fff1d56-a2bc-44bf-9370-410dde9bf254" containerName="manager" Jul 06 23:55:19.459739 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.459680 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fff1d56-a2bc-44bf-9370-410dde9bf254" containerName="manager" Jul 06 23:55:19.459801 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.459790 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fff1d56-a2bc-44bf-9370-410dde9bf254" containerName="manager" Jul 06 23:55:19.651968 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.651933 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr"] Jul 06 23:55:19.652124 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.652029 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.656353 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.656329 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g5l29\"" Jul 06 23:55:19.797982 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.797932 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.797982 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.797983 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.798220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.798053 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.798220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.798083 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.798220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.798103 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2eb3722-5915-4084-b152-86187682a2d3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.798220 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.798124 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnqrl\" (UniqueName: \"kubernetes.io/projected/e2eb3722-5915-4084-b152-86187682a2d3-kube-api-access-wnqrl\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.798361 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.798220 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2eb3722-5915-4084-b152-86187682a2d3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.798361 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.798241 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.798361 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.798262 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2eb3722-5915-4084-b152-86187682a2d3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.899727 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.899694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.899932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.899737 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.899932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.899756 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2eb3722-5915-4084-b152-86187682a2d3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.899932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.899783 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnqrl\" (UniqueName: \"kubernetes.io/projected/e2eb3722-5915-4084-b152-86187682a2d3-kube-api-access-wnqrl\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.899932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.899826 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2eb3722-5915-4084-b152-86187682a2d3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.899932 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.899853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.900222 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.899980 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2eb3722-5915-4084-b152-86187682a2d3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.900222 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.900073 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.900222 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.900099 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.900421 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.900396 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.900627 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.900462 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.900771 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.900711 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.900867 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.900821 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.901004 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.900981 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2eb3722-5915-4084-b152-86187682a2d3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.902498 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.902447 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2eb3722-5915-4084-b152-86187682a2d3-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.902849 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.902826 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2eb3722-5915-4084-b152-86187682a2d3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.908197 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.908172 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2eb3722-5915-4084-b152-86187682a2d3-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.908275 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.908224 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnqrl\" (UniqueName: \"kubernetes.io/projected/e2eb3722-5915-4084-b152-86187682a2d3-kube-api-access-wnqrl\") pod \"maas-default-gateway-openshift-default-56654dcf7-cbxpr\" (UID: \"e2eb3722-5915-4084-b152-86187682a2d3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:19.962557 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:19.962527 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:20.133340 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:20.133307 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr"] Jul 06 23:55:20.134655 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:55:20.134620 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2eb3722_5915_4084_b152_86187682a2d3.slice/crio-7415f40a35348f1c891fed8ab9a86c703c656276aff72cf76f1ed6e37af7c136 WatchSource:0}: Error finding container 7415f40a35348f1c891fed8ab9a86c703c656276aff72cf76f1ed6e37af7c136: Status 404 returned error can't find the container with id 7415f40a35348f1c891fed8ab9a86c703c656276aff72cf76f1ed6e37af7c136 Jul 06 23:55:20.136715 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:20.136683 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:55:20.136792 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:20.136752 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:55:20.136792 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:20.136782 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:55:20.323323 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:20.323287 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" event={"ID":"e2eb3722-5915-4084-b152-86187682a2d3","Type":"ContainerStarted","Data":"381fbfca2292681d3fb2c9572da936b3528f042245b9b1e1f382fc525a52ec57"} Jul 06 23:55:20.323536 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:20.323349 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" event={"ID":"e2eb3722-5915-4084-b152-86187682a2d3","Type":"ContainerStarted","Data":"7415f40a35348f1c891fed8ab9a86c703c656276aff72cf76f1ed6e37af7c136"} Jul 06 23:55:20.352398 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:20.352281 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" podStartSLOduration=1.352265404 podStartE2EDuration="1.352265404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:20.348796562 +0000 UTC m=+430.611468771" watchObservedRunningTime="2026-07-06 23:55:20.352265404 +0000 UTC m=+430.614937601" Jul 06 23:55:20.962669 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:20.962634 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:21.968792 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:21.968763 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:21.983012 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:21.982978 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:55:22.090697 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.090654 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:55:22.090872 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.090697 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:22.090872 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.090740 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:55:22.097661 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.097636 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 23:55:22.222681 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.222607 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/800bf725-b7a9-4b66-ac9c-bd9a13324f06-config-file\") pod \"limitador-limitador-54d6d988d5-d64r9\" (UID: \"800bf725-b7a9-4b66-ac9c-bd9a13324f06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:22.222681 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.222676 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njn6h\" (UniqueName: \"kubernetes.io/projected/800bf725-b7a9-4b66-ac9c-bd9a13324f06-kube-api-access-njn6h\") pod \"limitador-limitador-54d6d988d5-d64r9\" (UID: \"800bf725-b7a9-4b66-ac9c-bd9a13324f06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:22.324176 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.324138 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njn6h\" (UniqueName: \"kubernetes.io/projected/800bf725-b7a9-4b66-ac9c-bd9a13324f06-kube-api-access-njn6h\") pod \"limitador-limitador-54d6d988d5-d64r9\" (UID: \"800bf725-b7a9-4b66-ac9c-bd9a13324f06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:22.324340 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.324253 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/800bf725-b7a9-4b66-ac9c-bd9a13324f06-config-file\") pod \"limitador-limitador-54d6d988d5-d64r9\" (UID: \"800bf725-b7a9-4b66-ac9c-bd9a13324f06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:22.324921 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.324902 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/800bf725-b7a9-4b66-ac9c-bd9a13324f06-config-file\") pod \"limitador-limitador-54d6d988d5-d64r9\" (UID: \"800bf725-b7a9-4b66-ac9c-bd9a13324f06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:22.335549 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.335521 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:22.336637 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.336618 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:55:22.337208 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.337190 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njn6h\" (UniqueName: \"kubernetes.io/projected/800bf725-b7a9-4b66-ac9c-bd9a13324f06-kube-api-access-njn6h\") pod \"limitador-limitador-54d6d988d5-d64r9\" (UID: \"800bf725-b7a9-4b66-ac9c-bd9a13324f06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:22.401947 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.401918 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:22.536315 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.536274 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-lgcqk"] Jul 06 23:55:22.570063 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:55:22.570029 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800bf725_b7a9_4b66_ac9c_bd9a13324f06.slice/crio-6672f1dbba3f6736daaaa3c7a37edc9da0743fdda443ff36851bea623edc06b6 WatchSource:0}: Error finding container 6672f1dbba3f6736daaaa3c7a37edc9da0743fdda443ff36851bea623edc06b6: Status 404 returned error can't find the container with id 6672f1dbba3f6736daaaa3c7a37edc9da0743fdda443ff36851bea623edc06b6 Jul 06 23:55:22.576321 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.576300 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lgcqk"] Jul 06 23:55:22.576404 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.576329 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:55:22.576440 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.576418 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lgcqk" Jul 06 23:55:22.578167 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.578145 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ktbzk\"" Jul 06 23:55:22.728110 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.728072 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2kpf\" (UniqueName: \"kubernetes.io/projected/8f885c2f-4aa0-4b69-9434-62adf1b5d9be-kube-api-access-r2kpf\") pod \"authorino-75d556bd77-lgcqk\" (UID: \"8f885c2f-4aa0-4b69-9434-62adf1b5d9be\") " pod="kuadrant-system/authorino-75d556bd77-lgcqk" Jul 06 23:55:22.829128 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.829090 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2kpf\" (UniqueName: \"kubernetes.io/projected/8f885c2f-4aa0-4b69-9434-62adf1b5d9be-kube-api-access-r2kpf\") pod \"authorino-75d556bd77-lgcqk\" (UID: \"8f885c2f-4aa0-4b69-9434-62adf1b5d9be\") " pod="kuadrant-system/authorino-75d556bd77-lgcqk" Jul 06 23:55:22.839059 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.839038 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2kpf\" (UniqueName: \"kubernetes.io/projected/8f885c2f-4aa0-4b69-9434-62adf1b5d9be-kube-api-access-r2kpf\") pod \"authorino-75d556bd77-lgcqk\" (UID: \"8f885c2f-4aa0-4b69-9434-62adf1b5d9be\") " pod="kuadrant-system/authorino-75d556bd77-lgcqk" Jul 06 23:55:22.886332 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:22.886300 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lgcqk" Jul 06 23:55:23.074004 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:55:23.073970 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f885c2f_4aa0_4b69_9434_62adf1b5d9be.slice/crio-e77c9de78c1d3c52a2e4427bdbf8602a0a703145a6502be5ef0c4af6ed58a370 WatchSource:0}: Error finding container e77c9de78c1d3c52a2e4427bdbf8602a0a703145a6502be5ef0c4af6ed58a370: Status 404 returned error can't find the container with id e77c9de78c1d3c52a2e4427bdbf8602a0a703145a6502be5ef0c4af6ed58a370 Jul 06 23:55:23.076848 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:23.076715 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lgcqk"] Jul 06 23:55:23.345735 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:23.345694 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lgcqk" event={"ID":"8f885c2f-4aa0-4b69-9434-62adf1b5d9be","Type":"ContainerStarted","Data":"e77c9de78c1d3c52a2e4427bdbf8602a0a703145a6502be5ef0c4af6ed58a370"} Jul 06 23:55:23.348092 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:23.348061 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" event={"ID":"800bf725-b7a9-4b66-ac9c-bd9a13324f06","Type":"ContainerStarted","Data":"6672f1dbba3f6736daaaa3c7a37edc9da0743fdda443ff36851bea623edc06b6"} Jul 06 23:55:25.124989 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:25.124467 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-654cf8646-mttr5" podUID="0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" containerName="console" containerID="cri-o://12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990" gracePeriod=15 Jul 06 23:55:25.997678 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:25.997650 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:55:26.165313 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.165277 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-service-ca\") pod \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " Jul 06 23:55:26.165313 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.165315 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x75gq\" (UniqueName: \"kubernetes.io/projected/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-kube-api-access-x75gq\") pod \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " Jul 06 23:55:26.165884 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.165374 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-oauth-serving-cert\") pod \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " Jul 06 23:55:26.165884 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.165444 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-serving-cert\") pod \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " Jul 06 23:55:26.165884 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.165490 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-oauth-config\") pod \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " Jul 06 23:55:26.165884 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.165584 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-config\") pod \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " Jul 06 23:55:26.165884 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.165625 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-trusted-ca-bundle\") pod \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\" (UID: \"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8\") " Jul 06 23:55:26.165884 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.165749 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-service-ca" (OuterVolumeSpecName: "service-ca") pod "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" (UID: "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:55:26.166189 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.165946 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-service-ca\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:26.166189 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.166042 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-config" (OuterVolumeSpecName: "console-config") pod "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" (UID: "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:55:26.166302 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.166229 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" (UID: "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:55:26.166461 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.166407 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" (UID: "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:55:26.168163 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.168128 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" (UID: "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:55:26.168281 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.168214 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" (UID: "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:55:26.168281 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.168220 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-kube-api-access-x75gq" (OuterVolumeSpecName: "kube-api-access-x75gq") pod "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" (UID: "0f41d6c9-6d43-442c-901f-a6c89a4cf9c8"). InnerVolumeSpecName "kube-api-access-x75gq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:26.267125 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.267091 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-oauth-config\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:26.267125 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.267120 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-config\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:26.267125 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.267129 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-trusted-ca-bundle\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:26.267449 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.267138 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x75gq\" (UniqueName: \"kubernetes.io/projected/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-kube-api-access-x75gq\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:26.267449 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.267147 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-oauth-serving-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:26.267449 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.267159 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8-console-serving-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:26.372985 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.372945 2583 generic.go:356] "Generic (PLEG): container finished" podID="0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" containerID="12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990" exitCode=0 Jul 06 23:55:26.373160 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.373022 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-654cf8646-mttr5" Jul 06 23:55:26.373160 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.373031 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654cf8646-mttr5" event={"ID":"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8","Type":"ContainerDied","Data":"12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990"} Jul 06 23:55:26.373160 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.373082 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-654cf8646-mttr5" event={"ID":"0f41d6c9-6d43-442c-901f-a6c89a4cf9c8","Type":"ContainerDied","Data":"70c33bb6b744701e8bafa9215ebf6ccc8c2c23aaf27cf7623ed2db9e8cefd177"} Jul 06 23:55:26.373160 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.373103 2583 scope.go:117] "RemoveContainer" containerID="12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990" Jul 06 23:55:26.390428 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.390354 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-654cf8646-mttr5"] Jul 06 23:55:26.393496 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.393459 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-654cf8646-mttr5"] Jul 06 23:55:26.976296 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.976271 2583 scope.go:117] "RemoveContainer" containerID="12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990" Jul 06 23:55:26.976668 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:55:26.976648 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990\": container with ID starting with 12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990 not found: ID does not exist" containerID="12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990" Jul 06 23:55:26.976760 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:26.976675 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990"} err="failed to get container status \"12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990\": rpc error: code = NotFound desc = could not find container \"12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990\": container with ID starting with 12dc31631c0fbe67c12c7069171169c47c6cf0b9e258c0fe51689d4d3892d990 not found: ID does not exist" Jul 06 23:55:28.183737 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:28.183635 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" path="/var/lib/kubelet/pods/0f41d6c9-6d43-442c-901f-a6c89a4cf9c8/volumes" Jul 06 23:55:29.393274 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:29.393233 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lgcqk" event={"ID":"8f885c2f-4aa0-4b69-9434-62adf1b5d9be","Type":"ContainerStarted","Data":"0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b"} Jul 06 23:55:29.394755 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:29.394733 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" event={"ID":"800bf725-b7a9-4b66-ac9c-bd9a13324f06","Type":"ContainerStarted","Data":"712e92924b52dd162b9f4be6a1024daed1296d40df5fe03e543dcee67c616ce1"} Jul 06 23:55:29.394876 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:29.394859 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:29.409572 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:29.409531 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-lgcqk" podStartSLOduration=1.659215576 podStartE2EDuration="7.409516378s" totalImagesPullingTime="5.750300802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:29.407054851 +0000 UTC m=+439.669727047" watchObservedRunningTime="2026-07-06 23:55:29.409516378 +0000 UTC m=+439.672188573" Jul 06 23:55:29.427461 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:29.427412 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" podStartSLOduration=2.184206013 podStartE2EDuration="8.427394781s" totalImagesPullingTime="6.243188768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:29.426572958 +0000 UTC m=+439.689245154" watchObservedRunningTime="2026-07-06 23:55:29.427394781 +0000 UTC m=+439.690066976" Jul 06 23:55:40.402963 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:40.402884 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-d64r9" Jul 06 23:55:55.871392 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:55.871359 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mpkdw"] Jul 06 23:55:55.871807 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:55.871793 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" containerName="console" Jul 06 23:55:55.871851 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:55.871811 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" containerName="console" Jul 06 23:55:55.871904 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:55.871894 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f41d6c9-6d43-442c-901f-a6c89a4cf9c8" containerName="console" Jul 06 23:55:55.875145 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:55.875130 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" Jul 06 23:55:55.886440 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:55.886415 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mpkdw"] Jul 06 23:55:55.930410 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:55.930377 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtdqz\" (UniqueName: \"kubernetes.io/projected/ec971fc2-d5f4-4e5d-8159-42df292a4923-kube-api-access-jtdqz\") pod \"authorino-5d4b67c446-mpkdw\" (UID: \"ec971fc2-d5f4-4e5d-8159-42df292a4923\") " pod="kuadrant-system/authorino-5d4b67c446-mpkdw" Jul 06 23:55:56.031684 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.031648 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtdqz\" (UniqueName: \"kubernetes.io/projected/ec971fc2-d5f4-4e5d-8159-42df292a4923-kube-api-access-jtdqz\") pod \"authorino-5d4b67c446-mpkdw\" (UID: \"ec971fc2-d5f4-4e5d-8159-42df292a4923\") " pod="kuadrant-system/authorino-5d4b67c446-mpkdw" Jul 06 23:55:56.040083 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.040052 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtdqz\" (UniqueName: \"kubernetes.io/projected/ec971fc2-d5f4-4e5d-8159-42df292a4923-kube-api-access-jtdqz\") pod \"authorino-5d4b67c446-mpkdw\" (UID: \"ec971fc2-d5f4-4e5d-8159-42df292a4923\") " pod="kuadrant-system/authorino-5d4b67c446-mpkdw" Jul 06 23:55:56.095036 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.095004 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mpkdw"] Jul 06 23:55:56.095286 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.095274 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" Jul 06 23:55:56.123872 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.123814 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-794c94d844-8qt5b"] Jul 06 23:55:56.127601 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.127558 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794c94d844-8qt5b" Jul 06 23:55:56.140517 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.140493 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-794c94d844-8qt5b"] Jul 06 23:55:56.233837 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.233793 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h2nc\" (UniqueName: \"kubernetes.io/projected/9441c38b-bcda-4b78-8dfc-36e8d76c9bdc-kube-api-access-5h2nc\") pod \"authorino-794c94d844-8qt5b\" (UID: \"9441c38b-bcda-4b78-8dfc-36e8d76c9bdc\") " pod="kuadrant-system/authorino-794c94d844-8qt5b" Jul 06 23:55:56.238161 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.238131 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mpkdw"] Jul 06 23:55:56.243232 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:55:56.243197 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec971fc2_d5f4_4e5d_8159_42df292a4923.slice/crio-32d5728cf865b6930848be8f324380f1e42f9bea7f5056da15c7374a5243ceb2 WatchSource:0}: Error finding container 32d5728cf865b6930848be8f324380f1e42f9bea7f5056da15c7374a5243ceb2: Status 404 returned error can't find the container with id 32d5728cf865b6930848be8f324380f1e42f9bea7f5056da15c7374a5243ceb2 Jul 06 23:55:56.320950 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.320918 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-794c94d844-8qt5b"] Jul 06 23:55:56.321217 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:55:56.321196 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5h2nc], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-794c94d844-8qt5b" podUID="9441c38b-bcda-4b78-8dfc-36e8d76c9bdc" Jul 06 23:55:56.335127 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.335104 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h2nc\" (UniqueName: \"kubernetes.io/projected/9441c38b-bcda-4b78-8dfc-36e8d76c9bdc-kube-api-access-5h2nc\") pod \"authorino-794c94d844-8qt5b\" (UID: \"9441c38b-bcda-4b78-8dfc-36e8d76c9bdc\") " pod="kuadrant-system/authorino-794c94d844-8qt5b" Jul 06 23:55:56.348795 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.348769 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h2nc\" (UniqueName: \"kubernetes.io/projected/9441c38b-bcda-4b78-8dfc-36e8d76c9bdc-kube-api-access-5h2nc\") pod \"authorino-794c94d844-8qt5b\" (UID: \"9441c38b-bcda-4b78-8dfc-36e8d76c9bdc\") " pod="kuadrant-system/authorino-794c94d844-8qt5b" Jul 06 23:55:56.354113 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.354089 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7cd67c8ccb-6btnp"] Jul 06 23:55:56.358857 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.358837 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:55:56.362686 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.362668 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 23:55:56.370463 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.370444 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cd67c8ccb-6btnp"] Jul 06 23:55:56.436114 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.436040 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rng2w\" (UniqueName: \"kubernetes.io/projected/bf3c887c-4c4b-403b-a19f-4deea1485693-kube-api-access-rng2w\") pod \"authorino-7cd67c8ccb-6btnp\" (UID: \"bf3c887c-4c4b-403b-a19f-4deea1485693\") " pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:55:56.436114 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.436088 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf3c887c-4c4b-403b-a19f-4deea1485693-tls-cert\") pod \"authorino-7cd67c8ccb-6btnp\" (UID: \"bf3c887c-4c4b-403b-a19f-4deea1485693\") " pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:55:56.536837 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.536805 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rng2w\" (UniqueName: \"kubernetes.io/projected/bf3c887c-4c4b-403b-a19f-4deea1485693-kube-api-access-rng2w\") pod \"authorino-7cd67c8ccb-6btnp\" (UID: \"bf3c887c-4c4b-403b-a19f-4deea1485693\") " pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:55:56.537035 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.536847 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf3c887c-4c4b-403b-a19f-4deea1485693-tls-cert\") pod \"authorino-7cd67c8ccb-6btnp\" (UID: \"bf3c887c-4c4b-403b-a19f-4deea1485693\") " pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:55:56.539228 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.539209 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf3c887c-4c4b-403b-a19f-4deea1485693-tls-cert\") pod \"authorino-7cd67c8ccb-6btnp\" (UID: \"bf3c887c-4c4b-403b-a19f-4deea1485693\") " pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:55:56.546017 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.545994 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rng2w\" (UniqueName: \"kubernetes.io/projected/bf3c887c-4c4b-403b-a19f-4deea1485693-kube-api-access-rng2w\") pod \"authorino-7cd67c8ccb-6btnp\" (UID: \"bf3c887c-4c4b-403b-a19f-4deea1485693\") " pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:55:56.550720 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.550694 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" event={"ID":"ec971fc2-d5f4-4e5d-8159-42df292a4923","Type":"ContainerStarted","Data":"32d5728cf865b6930848be8f324380f1e42f9bea7f5056da15c7374a5243ceb2"} Jul 06 23:55:56.550720 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.550711 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794c94d844-8qt5b" Jul 06 23:55:56.557331 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.557314 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794c94d844-8qt5b" Jul 06 23:55:56.638147 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.638128 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h2nc\" (UniqueName: \"kubernetes.io/projected/9441c38b-bcda-4b78-8dfc-36e8d76c9bdc-kube-api-access-5h2nc\") pod \"9441c38b-bcda-4b78-8dfc-36e8d76c9bdc\" (UID: \"9441c38b-bcda-4b78-8dfc-36e8d76c9bdc\") " Jul 06 23:55:56.639935 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.639910 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9441c38b-bcda-4b78-8dfc-36e8d76c9bdc-kube-api-access-5h2nc" (OuterVolumeSpecName: "kube-api-access-5h2nc") pod "9441c38b-bcda-4b78-8dfc-36e8d76c9bdc" (UID: "9441c38b-bcda-4b78-8dfc-36e8d76c9bdc"). InnerVolumeSpecName "kube-api-access-5h2nc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:56.668720 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.668693 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:55:56.738910 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.738880 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h2nc\" (UniqueName: \"kubernetes.io/projected/9441c38b-bcda-4b78-8dfc-36e8d76c9bdc-kube-api-access-5h2nc\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:56.824021 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:56.823999 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7cd67c8ccb-6btnp"] Jul 06 23:55:56.825661 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:55:56.825629 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf3c887c_4c4b_403b_a19f_4deea1485693.slice/crio-247efa079b4bf321a3043f6c0a5ab85fb10b2d25a64af773ead3d65ed2321ac3 WatchSource:0}: Error finding container 247efa079b4bf321a3043f6c0a5ab85fb10b2d25a64af773ead3d65ed2321ac3: Status 404 returned error can't find the container with id 247efa079b4bf321a3043f6c0a5ab85fb10b2d25a64af773ead3d65ed2321ac3 Jul 06 23:55:57.559282 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.559181 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" event={"ID":"bf3c887c-4c4b-403b-a19f-4deea1485693","Type":"ContainerStarted","Data":"9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0"} Jul 06 23:55:57.559282 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.559240 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" event={"ID":"bf3c887c-4c4b-403b-a19f-4deea1485693","Type":"ContainerStarted","Data":"247efa079b4bf321a3043f6c0a5ab85fb10b2d25a64af773ead3d65ed2321ac3"} Jul 06 23:55:57.560735 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.560706 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" event={"ID":"ec971fc2-d5f4-4e5d-8159-42df292a4923","Type":"ContainerStarted","Data":"087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f"} Jul 06 23:55:57.560829 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.560738 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794c94d844-8qt5b" Jul 06 23:55:57.560829 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.560748 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" podUID="ec971fc2-d5f4-4e5d-8159-42df292a4923" containerName="authorino" containerID="cri-o://087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f" gracePeriod=30 Jul 06 23:55:57.577250 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.577209 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" podStartSLOduration=1.117573313 podStartE2EDuration="1.577194881s" totalImagesPullingTime="459.621568ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:57.574832071 +0000 UTC m=+467.837504267" watchObservedRunningTime="2026-07-06 23:55:57.577194881 +0000 UTC m=+467.839867135" Jul 06 23:55:57.601619 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.601591 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-794c94d844-8qt5b"] Jul 06 23:55:57.610215 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.610194 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-794c94d844-8qt5b"] Jul 06 23:55:57.618222 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.618198 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lgcqk"] Jul 06 23:55:57.618375 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.618355 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-lgcqk" podUID="8f885c2f-4aa0-4b69-9434-62adf1b5d9be" containerName="authorino" containerID="cri-o://0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b" gracePeriod=30 Jul 06 23:55:57.682002 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.681941 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" podStartSLOduration=2.327293931 podStartE2EDuration="2.681921269s" totalImagesPullingTime="354.627338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:55:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:55:57.645052823 +0000 UTC m=+467.907725019" watchObservedRunningTime="2026-07-06 23:55:57.681921269 +0000 UTC m=+467.944593463" Jul 06 23:55:57.871027 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.871001 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" Jul 06 23:55:57.880744 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.880721 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lgcqk" Jul 06 23:55:57.950969 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.950935 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2kpf\" (UniqueName: \"kubernetes.io/projected/8f885c2f-4aa0-4b69-9434-62adf1b5d9be-kube-api-access-r2kpf\") pod \"8f885c2f-4aa0-4b69-9434-62adf1b5d9be\" (UID: \"8f885c2f-4aa0-4b69-9434-62adf1b5d9be\") " Jul 06 23:55:57.951135 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.951022 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtdqz\" (UniqueName: \"kubernetes.io/projected/ec971fc2-d5f4-4e5d-8159-42df292a4923-kube-api-access-jtdqz\") pod \"ec971fc2-d5f4-4e5d-8159-42df292a4923\" (UID: \"ec971fc2-d5f4-4e5d-8159-42df292a4923\") " Jul 06 23:55:57.953108 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.953076 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f885c2f-4aa0-4b69-9434-62adf1b5d9be-kube-api-access-r2kpf" (OuterVolumeSpecName: "kube-api-access-r2kpf") pod "8f885c2f-4aa0-4b69-9434-62adf1b5d9be" (UID: "8f885c2f-4aa0-4b69-9434-62adf1b5d9be"). InnerVolumeSpecName "kube-api-access-r2kpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:57.953108 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:57.953100 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec971fc2-d5f4-4e5d-8159-42df292a4923-kube-api-access-jtdqz" (OuterVolumeSpecName: "kube-api-access-jtdqz") pod "ec971fc2-d5f4-4e5d-8159-42df292a4923" (UID: "ec971fc2-d5f4-4e5d-8159-42df292a4923"). InnerVolumeSpecName "kube-api-access-jtdqz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:55:58.051893 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.051861 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtdqz\" (UniqueName: \"kubernetes.io/projected/ec971fc2-d5f4-4e5d-8159-42df292a4923-kube-api-access-jtdqz\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:58.051893 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.051894 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2kpf\" (UniqueName: \"kubernetes.io/projected/8f885c2f-4aa0-4b69-9434-62adf1b5d9be-kube-api-access-r2kpf\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:55:58.182403 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.182321 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9441c38b-bcda-4b78-8dfc-36e8d76c9bdc" path="/var/lib/kubelet/pods/9441c38b-bcda-4b78-8dfc-36e8d76c9bdc/volumes" Jul 06 23:55:58.567512 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.567458 2583 generic.go:356] "Generic (PLEG): container finished" podID="ec971fc2-d5f4-4e5d-8159-42df292a4923" containerID="087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f" exitCode=0 Jul 06 23:55:58.567947 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.567520 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" Jul 06 23:55:58.567947 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.567541 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" event={"ID":"ec971fc2-d5f4-4e5d-8159-42df292a4923","Type":"ContainerDied","Data":"087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f"} Jul 06 23:55:58.567947 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.567598 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mpkdw" event={"ID":"ec971fc2-d5f4-4e5d-8159-42df292a4923","Type":"ContainerDied","Data":"32d5728cf865b6930848be8f324380f1e42f9bea7f5056da15c7374a5243ceb2"} Jul 06 23:55:58.567947 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.567618 2583 scope.go:117] "RemoveContainer" containerID="087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f" Jul 06 23:55:58.569138 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.569110 2583 generic.go:356] "Generic (PLEG): container finished" podID="8f885c2f-4aa0-4b69-9434-62adf1b5d9be" containerID="0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b" exitCode=0 Jul 06 23:55:58.569251 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.569176 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-lgcqk" Jul 06 23:55:58.569309 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.569169 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lgcqk" event={"ID":"8f885c2f-4aa0-4b69-9434-62adf1b5d9be","Type":"ContainerDied","Data":"0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b"} Jul 06 23:55:58.569309 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.569282 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-lgcqk" event={"ID":"8f885c2f-4aa0-4b69-9434-62adf1b5d9be","Type":"ContainerDied","Data":"e77c9de78c1d3c52a2e4427bdbf8602a0a703145a6502be5ef0c4af6ed58a370"} Jul 06 23:55:58.576699 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.576683 2583 scope.go:117] "RemoveContainer" containerID="087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f" Jul 06 23:55:58.576941 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:55:58.576922 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f\": container with ID starting with 087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f not found: ID does not exist" containerID="087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f" Jul 06 23:55:58.576989 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.576949 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f"} err="failed to get container status \"087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f\": rpc error: code = NotFound desc = could not find container \"087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f\": container with ID starting with 087dfb3a13cedd1dde5cb8a32cd5d400a4449fab698b823cd2971b706b8e4c8f not found: ID does not exist" Jul 06 23:55:58.576989 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.576966 2583 scope.go:117] "RemoveContainer" containerID="0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b" Jul 06 23:55:58.584919 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.584905 2583 scope.go:117] "RemoveContainer" containerID="0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b" Jul 06 23:55:58.585144 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:55:58.585126 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b\": container with ID starting with 0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b not found: ID does not exist" containerID="0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b" Jul 06 23:55:58.585242 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.585150 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b"} err="failed to get container status \"0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b\": rpc error: code = NotFound desc = could not find container \"0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b\": container with ID starting with 0f5f6f4ebb14c9999e99befcbf8d61d96ab56ca34a5605df7ef464dd3e27465b not found: ID does not exist" Jul 06 23:55:58.588454 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.588433 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mpkdw"] Jul 06 23:55:58.592459 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.592440 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mpkdw"] Jul 06 23:55:58.602650 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.602631 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lgcqk"] Jul 06 23:55:58.611391 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:55:58.611375 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-lgcqk"] Jul 06 23:56:00.181755 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:00.181724 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f885c2f-4aa0-4b69-9434-62adf1b5d9be" path="/var/lib/kubelet/pods/8f885c2f-4aa0-4b69-9434-62adf1b5d9be/volumes" Jul 06 23:56:00.182205 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:00.182022 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec971fc2-d5f4-4e5d-8159-42df292a4923" path="/var/lib/kubelet/pods/ec971fc2-d5f4-4e5d-8159-42df292a4923/volumes" Jul 06 23:56:38.806920 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:38.806884 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:56:46.859564 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:46.859529 2583 generic.go:356] "Generic (PLEG): container finished" podID="e2eb3722-5915-4084-b152-86187682a2d3" containerID="381fbfca2292681d3fb2c9572da936b3528f042245b9b1e1f382fc525a52ec57" exitCode=0 Jul 06 23:56:46.860002 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:46.859613 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" event={"ID":"e2eb3722-5915-4084-b152-86187682a2d3","Type":"ContainerDied","Data":"381fbfca2292681d3fb2c9572da936b3528f042245b9b1e1f382fc525a52ec57"} Jul 06 23:56:46.860058 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:46.860019 2583 scope.go:117] "RemoveContainer" containerID="381fbfca2292681d3fb2c9572da936b3528f042245b9b1e1f382fc525a52ec57" Jul 06 23:56:46.861042 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:46.861016 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:56:46.861092 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:46.861082 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:56:46.861133 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:46.861113 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 06 23:56:47.866995 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:47.866959 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" event={"ID":"e2eb3722-5915-4084-b152-86187682a2d3","Type":"ContainerStarted","Data":"3f64717a80efcb3c932b7a82350de18a5d5e615bb4aa384fdd02624d002b6f52"} Jul 06 23:56:47.963123 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:47.963080 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:56:47.964234 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:47.964203 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cbxpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 06 23:56:47.964354 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:47.964272 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" podUID="e2eb3722-5915-4084-b152-86187682a2d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 06 23:56:48.963755 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:48.963712 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cbxpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 06 23:56:48.964157 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:48.963788 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" podUID="e2eb3722-5915-4084-b152-86187682a2d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 06 23:56:49.963700 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:49.963661 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cbxpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 06 23:56:49.963880 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:49.963734 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" podUID="e2eb3722-5915-4084-b152-86187682a2d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 06 23:56:50.963349 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:50.963309 2583 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-cbxpr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 06 23:56:50.963546 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:50.963363 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" podUID="e2eb3722-5915-4084-b152-86187682a2d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 06 23:56:51.967110 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:51.967076 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:56:51.967569 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:51.967467 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:56:51.968127 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:51.968110 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-cbxpr" Jul 06 23:56:59.499518 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:56:59.499460 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:57:10.008747 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:57:10.008661 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:57:23.200609 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:57:23.200568 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:57:37.841982 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:57:37.841941 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:57:43.806750 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:57:43.806710 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:57:48.907629 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:57:48.907577 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:58:47.068046 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.067960 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-785697d47c-j9vjc"] Jul 06 23:58:47.068655 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.068359 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec971fc2-d5f4-4e5d-8159-42df292a4923" containerName="authorino" Jul 06 23:58:47.068655 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.068370 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec971fc2-d5f4-4e5d-8159-42df292a4923" containerName="authorino" Jul 06 23:58:47.068655 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.068385 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f885c2f-4aa0-4b69-9434-62adf1b5d9be" containerName="authorino" Jul 06 23:58:47.068655 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.068390 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f885c2f-4aa0-4b69-9434-62adf1b5d9be" containerName="authorino" Jul 06 23:58:47.068655 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.068462 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f885c2f-4aa0-4b69-9434-62adf1b5d9be" containerName="authorino" Jul 06 23:58:47.068655 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.068491 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec971fc2-d5f4-4e5d-8159-42df292a4923" containerName="authorino" Jul 06 23:58:47.071368 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.071340 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-785697d47c-j9vjc" Jul 06 23:58:47.080104 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.080080 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-785697d47c-j9vjc"] Jul 06 23:58:47.196903 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.196871 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm27v\" (UniqueName: \"kubernetes.io/projected/2445a1d8-2828-4deb-96e4-1f0004c17837-kube-api-access-dm27v\") pod \"authorino-785697d47c-j9vjc\" (UID: \"2445a1d8-2828-4deb-96e4-1f0004c17837\") " pod="kuadrant-system/authorino-785697d47c-j9vjc" Jul 06 23:58:47.197073 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.196942 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2445a1d8-2828-4deb-96e4-1f0004c17837-tls-cert\") pod \"authorino-785697d47c-j9vjc\" (UID: \"2445a1d8-2828-4deb-96e4-1f0004c17837\") " pod="kuadrant-system/authorino-785697d47c-j9vjc" Jul 06 23:58:47.298428 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.298393 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm27v\" (UniqueName: \"kubernetes.io/projected/2445a1d8-2828-4deb-96e4-1f0004c17837-kube-api-access-dm27v\") pod \"authorino-785697d47c-j9vjc\" (UID: \"2445a1d8-2828-4deb-96e4-1f0004c17837\") " pod="kuadrant-system/authorino-785697d47c-j9vjc" Jul 06 23:58:47.298648 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.298503 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2445a1d8-2828-4deb-96e4-1f0004c17837-tls-cert\") pod \"authorino-785697d47c-j9vjc\" (UID: \"2445a1d8-2828-4deb-96e4-1f0004c17837\") " pod="kuadrant-system/authorino-785697d47c-j9vjc" Jul 06 23:58:47.301007 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.300979 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2445a1d8-2828-4deb-96e4-1f0004c17837-tls-cert\") pod \"authorino-785697d47c-j9vjc\" (UID: \"2445a1d8-2828-4deb-96e4-1f0004c17837\") " pod="kuadrant-system/authorino-785697d47c-j9vjc" Jul 06 23:58:47.305947 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.305923 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm27v\" (UniqueName: \"kubernetes.io/projected/2445a1d8-2828-4deb-96e4-1f0004c17837-kube-api-access-dm27v\") pod \"authorino-785697d47c-j9vjc\" (UID: \"2445a1d8-2828-4deb-96e4-1f0004c17837\") " pod="kuadrant-system/authorino-785697d47c-j9vjc" Jul 06 23:58:47.381187 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.381104 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-785697d47c-j9vjc" Jul 06 23:58:47.530052 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.530024 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-785697d47c-j9vjc"] Jul 06 23:58:47.531701 ip-10-0-137-188 kubenswrapper[2583]: W0706 23:58:47.531675 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2445a1d8_2828_4deb_96e4_1f0004c17837.slice/crio-eee5d342e1629cf09680eeac8f0fc8fbeab6962249e73246c0af580ba6c45cc0 WatchSource:0}: Error finding container eee5d342e1629cf09680eeac8f0fc8fbeab6962249e73246c0af580ba6c45cc0: Status 404 returned error can't find the container with id eee5d342e1629cf09680eeac8f0fc8fbeab6962249e73246c0af580ba6c45cc0 Jul 06 23:58:47.553904 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:47.553874 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-785697d47c-j9vjc" event={"ID":"2445a1d8-2828-4deb-96e4-1f0004c17837","Type":"ContainerStarted","Data":"eee5d342e1629cf09680eeac8f0fc8fbeab6962249e73246c0af580ba6c45cc0"} Jul 06 23:58:48.562905 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:48.562853 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-785697d47c-j9vjc" event={"ID":"2445a1d8-2828-4deb-96e4-1f0004c17837","Type":"ContainerStarted","Data":"2483e22b6a6724b490a96bfc52f5305725237f86b0d969c155cf4d758651ca14"} Jul 06 23:58:48.588587 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:48.588527 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-785697d47c-j9vjc" podStartSLOduration=1.260799528 podStartE2EDuration="1.588502472s" totalImagesPullingTime="327.702944ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:58:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:58:48.587111342 +0000 UTC m=+638.849783540" watchObservedRunningTime="2026-07-06 23:58:48.588502472 +0000 UTC m=+638.851174668" Jul 06 23:58:48.622916 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:48.622878 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7cd67c8ccb-6btnp"] Jul 06 23:58:48.623133 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:48.623103 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" podUID="bf3c887c-4c4b-403b-a19f-4deea1485693" containerName="authorino" containerID="cri-o://9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0" gracePeriod=30 Jul 06 23:58:48.885762 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:48.885738 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:58:48.909352 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:48.909120 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rng2w\" (UniqueName: \"kubernetes.io/projected/bf3c887c-4c4b-403b-a19f-4deea1485693-kube-api-access-rng2w\") pod \"bf3c887c-4c4b-403b-a19f-4deea1485693\" (UID: \"bf3c887c-4c4b-403b-a19f-4deea1485693\") " Jul 06 23:58:48.909661 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:48.909640 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf3c887c-4c4b-403b-a19f-4deea1485693-tls-cert\") pod \"bf3c887c-4c4b-403b-a19f-4deea1485693\" (UID: \"bf3c887c-4c4b-403b-a19f-4deea1485693\") " Jul 06 23:58:48.912005 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:48.911971 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf3c887c-4c4b-403b-a19f-4deea1485693-kube-api-access-rng2w" (OuterVolumeSpecName: "kube-api-access-rng2w") pod "bf3c887c-4c4b-403b-a19f-4deea1485693" (UID: "bf3c887c-4c4b-403b-a19f-4deea1485693"). InnerVolumeSpecName "kube-api-access-rng2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:58:48.920642 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:48.920605 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf3c887c-4c4b-403b-a19f-4deea1485693-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "bf3c887c-4c4b-403b-a19f-4deea1485693" (UID: "bf3c887c-4c4b-403b-a19f-4deea1485693"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:58:49.011452 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.011412 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rng2w\" (UniqueName: \"kubernetes.io/projected/bf3c887c-4c4b-403b-a19f-4deea1485693-kube-api-access-rng2w\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:58:49.011452 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.011452 2583 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf3c887c-4c4b-403b-a19f-4deea1485693-tls-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 06 23:58:49.571922 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.571836 2583 generic.go:356] "Generic (PLEG): container finished" podID="bf3c887c-4c4b-403b-a19f-4deea1485693" containerID="9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0" exitCode=0 Jul 06 23:58:49.571922 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.571892 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" Jul 06 23:58:49.572442 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.571919 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" event={"ID":"bf3c887c-4c4b-403b-a19f-4deea1485693","Type":"ContainerDied","Data":"9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0"} Jul 06 23:58:49.572442 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.571957 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7cd67c8ccb-6btnp" event={"ID":"bf3c887c-4c4b-403b-a19f-4deea1485693","Type":"ContainerDied","Data":"247efa079b4bf321a3043f6c0a5ab85fb10b2d25a64af773ead3d65ed2321ac3"} Jul 06 23:58:49.572442 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.571975 2583 scope.go:117] "RemoveContainer" containerID="9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0" Jul 06 23:58:49.581202 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.581183 2583 scope.go:117] "RemoveContainer" containerID="9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0" Jul 06 23:58:49.581489 ip-10-0-137-188 kubenswrapper[2583]: E0706 23:58:49.581456 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0\": container with ID starting with 9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0 not found: ID does not exist" containerID="9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0" Jul 06 23:58:49.581585 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.581505 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0"} err="failed to get container status \"9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0\": rpc error: code = NotFound desc = could not find container \"9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0\": container with ID starting with 9691774af3ab24b4a8345237a36dcc165468751e91ad53bd60168a1abdb97da0 not found: ID does not exist" Jul 06 23:58:49.597894 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.597865 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7cd67c8ccb-6btnp"] Jul 06 23:58:49.604238 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:49.604212 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7cd67c8ccb-6btnp"] Jul 06 23:58:50.182822 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:58:50.182790 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf3c887c-4c4b-403b-a19f-4deea1485693" path="/var/lib/kubelet/pods/bf3c887c-4c4b-403b-a19f-4deea1485693/volumes" Jul 06 23:59:15.806422 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:59:15.806382 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:59:25.692879 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:59:25.692840 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:59:35.600912 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:59:35.600872 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:59:45.803886 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:59:45.803849 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 06 23:59:55.898956 ip-10-0-137-188 kubenswrapper[2583]: I0706 23:59:55.898914 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:00:00.171418 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.171373 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723040-njdwh"] Jul 07 00:00:00.171998 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.171977 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf3c887c-4c4b-403b-a19f-4deea1485693" containerName="authorino" Jul 07 00:00:00.172085 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.172001 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf3c887c-4c4b-403b-a19f-4deea1485693" containerName="authorino" Jul 07 00:00:00.172177 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.172164 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf3c887c-4c4b-403b-a19f-4deea1485693" containerName="authorino" Jul 07 00:00:00.176197 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.176119 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" Jul 07 00:00:00.179583 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.179558 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-7qrr6\"" Jul 07 00:00:00.188933 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.188899 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29723040-q5fxn"] Jul 07 00:00:00.193413 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.193314 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723040-njdwh"] Jul 07 00:00:00.193582 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.193456 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:00.196043 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.195801 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\"" Jul 07 00:00:00.196654 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.196626 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-tvnx7\"" Jul 07 00:00:00.198160 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.198138 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29723040-q5fxn"] Jul 07 00:00:00.264559 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.264527 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7k2k\" (UniqueName: \"kubernetes.io/projected/4a50d927-75ad-49c3-a48a-f72ff3f6027f-kube-api-access-x7k2k\") pod \"maas-api-key-cleanup-29723040-njdwh\" (UID: \"4a50d927-75ad-49c3-a48a-f72ff3f6027f\") " pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" Jul 07 00:00:00.264559 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.264563 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-serviceca\") pod \"image-pruner-29723040-q5fxn\" (UID: \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\") " pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:00.264834 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.264602 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzc25\" (UniqueName: \"kubernetes.io/projected/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-kube-api-access-lzc25\") pod \"image-pruner-29723040-q5fxn\" (UID: \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\") " pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:00.365397 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.365363 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7k2k\" (UniqueName: \"kubernetes.io/projected/4a50d927-75ad-49c3-a48a-f72ff3f6027f-kube-api-access-x7k2k\") pod \"maas-api-key-cleanup-29723040-njdwh\" (UID: \"4a50d927-75ad-49c3-a48a-f72ff3f6027f\") " pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" Jul 07 00:00:00.365397 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.365398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-serviceca\") pod \"image-pruner-29723040-q5fxn\" (UID: \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\") " pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:00.365628 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.365422 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzc25\" (UniqueName: \"kubernetes.io/projected/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-kube-api-access-lzc25\") pod \"image-pruner-29723040-q5fxn\" (UID: \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\") " pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:00.366029 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.366009 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-serviceca\") pod \"image-pruner-29723040-q5fxn\" (UID: \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\") " pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:00.376299 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.376271 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7k2k\" (UniqueName: \"kubernetes.io/projected/4a50d927-75ad-49c3-a48a-f72ff3f6027f-kube-api-access-x7k2k\") pod \"maas-api-key-cleanup-29723040-njdwh\" (UID: \"4a50d927-75ad-49c3-a48a-f72ff3f6027f\") " pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" Jul 07 00:00:00.376444 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.376421 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzc25\" (UniqueName: \"kubernetes.io/projected/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-kube-api-access-lzc25\") pod \"image-pruner-29723040-q5fxn\" (UID: \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\") " pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:00.511261 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.511227 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" Jul 07 00:00:00.516214 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.516189 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:00.666634 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.666605 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723040-njdwh"] Jul 07 00:00:00.669232 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:00:00.669204 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a50d927_75ad_49c3_a48a_f72ff3f6027f.slice/crio-978c27367fb5cbd84e4bd2413518a827e697af13439f289235ebdf03f95c8e64 WatchSource:0}: Error finding container 978c27367fb5cbd84e4bd2413518a827e697af13439f289235ebdf03f95c8e64: Status 404 returned error can't find the container with id 978c27367fb5cbd84e4bd2413518a827e697af13439f289235ebdf03f95c8e64 Jul 07 00:00:00.671254 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.671235 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:00:00.700023 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.699996 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29723040-q5fxn"] Jul 07 00:00:00.701894 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:00:00.701864 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a41f65f_ca12_4745_8d9f_0c86e5e081cb.slice/crio-9f2e487070788cb45c411c825cf93a61fe44d7c4fb36ac9947dcb9b8c794e00c WatchSource:0}: Error finding container 9f2e487070788cb45c411c825cf93a61fe44d7c4fb36ac9947dcb9b8c794e00c: Status 404 returned error can't find the container with id 9f2e487070788cb45c411c825cf93a61fe44d7c4fb36ac9947dcb9b8c794e00c Jul 07 00:00:00.990457 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.990416 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-q5fxn" event={"ID":"3a41f65f-ca12-4745-8d9f-0c86e5e081cb","Type":"ContainerStarted","Data":"26a8d984bfd4356b3f4cbd0c1e315d0f07168d8004eb8eebb4811505612eeff3"} Jul 07 00:00:00.990662 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.990502 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-q5fxn" event={"ID":"3a41f65f-ca12-4745-8d9f-0c86e5e081cb","Type":"ContainerStarted","Data":"9f2e487070788cb45c411c825cf93a61fe44d7c4fb36ac9947dcb9b8c794e00c"} Jul 07 00:00:00.991625 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:00.991595 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" event={"ID":"4a50d927-75ad-49c3-a48a-f72ff3f6027f","Type":"ContainerStarted","Data":"978c27367fb5cbd84e4bd2413518a827e697af13439f289235ebdf03f95c8e64"} Jul 07 00:00:01.007693 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:01.007634 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-pruner-29723040-q5fxn" podStartSLOduration=1.007619266 podStartE2EDuration="1.007619266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:00:01.007279748 +0000 UTC m=+711.269951955" watchObservedRunningTime="2026-07-07 00:00:01.007619266 +0000 UTC m=+711.270291439" Jul 07 00:00:01.998669 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:01.998589 2583 generic.go:356] "Generic (PLEG): container finished" podID="3a41f65f-ca12-4745-8d9f-0c86e5e081cb" containerID="26a8d984bfd4356b3f4cbd0c1e315d0f07168d8004eb8eebb4811505612eeff3" exitCode=0 Jul 07 00:00:01.999032 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:01.998675 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-q5fxn" event={"ID":"3a41f65f-ca12-4745-8d9f-0c86e5e081cb","Type":"ContainerDied","Data":"26a8d984bfd4356b3f4cbd0c1e315d0f07168d8004eb8eebb4811505612eeff3"} Jul 07 00:00:03.146205 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:03.146180 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:03.293719 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:03.293623 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lzc25\" (UniqueName: \"kubernetes.io/projected/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-kube-api-access-lzc25\") pod \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\" (UID: \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\") " Jul 07 00:00:03.293719 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:03.293719 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-serviceca\") pod \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\" (UID: \"3a41f65f-ca12-4745-8d9f-0c86e5e081cb\") " Jul 07 00:00:03.294070 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:03.294042 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-serviceca" (OuterVolumeSpecName: "serviceca") pod "3a41f65f-ca12-4745-8d9f-0c86e5e081cb" (UID: "3a41f65f-ca12-4745-8d9f-0c86e5e081cb"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:00:03.295866 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:03.295837 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-kube-api-access-lzc25" (OuterVolumeSpecName: "kube-api-access-lzc25") pod "3a41f65f-ca12-4745-8d9f-0c86e5e081cb" (UID: "3a41f65f-ca12-4745-8d9f-0c86e5e081cb"). InnerVolumeSpecName "kube-api-access-lzc25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:00:03.395058 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:03.395029 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lzc25\" (UniqueName: \"kubernetes.io/projected/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-kube-api-access-lzc25\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:00:03.395058 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:03.395053 2583 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3a41f65f-ca12-4745-8d9f-0c86e5e081cb-serviceca\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:00:04.017402 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:04.017361 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29723040-q5fxn" event={"ID":"3a41f65f-ca12-4745-8d9f-0c86e5e081cb","Type":"ContainerDied","Data":"9f2e487070788cb45c411c825cf93a61fe44d7c4fb36ac9947dcb9b8c794e00c"} Jul 07 00:00:04.017402 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:04.017387 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29723040-q5fxn" Jul 07 00:00:04.017609 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:04.017449 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f2e487070788cb45c411c825cf93a61fe44d7c4fb36ac9947dcb9b8c794e00c" Jul 07 00:00:23.015737 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:23.015700 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:00:27.163486 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:27.163439 2583 generic.go:356] "Generic (PLEG): container finished" podID="4a50d927-75ad-49c3-a48a-f72ff3f6027f" containerID="cdefa2f7ee3b59457e01c7b04869ff186c07b621f6ae5086eaa0616b3d4a7c98" exitCode=0 Jul 07 00:00:27.163900 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:27.163512 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" event={"ID":"4a50d927-75ad-49c3-a48a-f72ff3f6027f","Type":"ContainerDied","Data":"cdefa2f7ee3b59457e01c7b04869ff186c07b621f6ae5086eaa0616b3d4a7c98"} Jul 07 00:00:28.301946 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:28.301923 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" Jul 07 00:00:28.404957 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:28.404929 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7k2k\" (UniqueName: \"kubernetes.io/projected/4a50d927-75ad-49c3-a48a-f72ff3f6027f-kube-api-access-x7k2k\") pod \"4a50d927-75ad-49c3-a48a-f72ff3f6027f\" (UID: \"4a50d927-75ad-49c3-a48a-f72ff3f6027f\") " Jul 07 00:00:28.406975 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:28.406955 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a50d927-75ad-49c3-a48a-f72ff3f6027f-kube-api-access-x7k2k" (OuterVolumeSpecName: "kube-api-access-x7k2k") pod "4a50d927-75ad-49c3-a48a-f72ff3f6027f" (UID: "4a50d927-75ad-49c3-a48a-f72ff3f6027f"). InnerVolumeSpecName "kube-api-access-x7k2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:00:28.505838 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:28.505806 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7k2k\" (UniqueName: \"kubernetes.io/projected/4a50d927-75ad-49c3-a48a-f72ff3f6027f-kube-api-access-x7k2k\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:00:29.176729 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:29.176700 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" Jul 07 00:00:29.176893 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:29.176737 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723040-njdwh" event={"ID":"4a50d927-75ad-49c3-a48a-f72ff3f6027f","Type":"ContainerDied","Data":"978c27367fb5cbd84e4bd2413518a827e697af13439f289235ebdf03f95c8e64"} Jul 07 00:00:29.176893 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:29.176781 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="978c27367fb5cbd84e4bd2413518a827e697af13439f289235ebdf03f95c8e64" Jul 07 00:00:34.098504 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:00:34.098458 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:01:08.296753 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:01:08.296720 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:01:18.003622 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:01:18.003573 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:01:27.490536 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:01:27.490502 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:01:34.896926 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:01:34.896842 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:01:46.103281 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:01:46.103156 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:01:55.402080 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:01:55.402043 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:02:06.795881 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:02:06.795839 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:02:16.195388 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:02:16.195353 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:02:38.697849 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:02:38.697819 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:02:50.995200 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:02:50.995164 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:03:13.598560 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:03:13.598521 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:03:23.299129 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:03:23.299093 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:03:38.890111 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:03:38.890072 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:03:49.399643 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:03:49.399593 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:03:59.899077 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:03:59.899041 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:04:09.388791 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:04:09.388755 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:04:18.191487 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:04:18.191444 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:06:07.498101 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:06:07.498069 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:06:18.904959 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:06:18.904921 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:06:27.601258 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:06:27.601222 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:06:38.693818 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:06:38.693779 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:06:50.092893 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:06:50.092850 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:07:00.500301 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:07:00.500264 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:07:09.193628 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:07:09.193587 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:07:19.693041 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:07:19.692996 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:07:31.395833 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:07:31.395791 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:07:40.799652 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:07:40.799559 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:07:50.890534 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:07:50.890493 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:08:12.703740 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:08:12.703698 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:08:21.700873 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:08:21.700836 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:08:33.495530 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:08:33.495496 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:08:42.591036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:08:42.590994 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:08:52.592996 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:08:52.592959 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:09:02.004364 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:09:02.004269 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:09:14.199908 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:09:14.199872 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:09:23.000714 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:09:23.000677 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:09:32.598348 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:09:32.598304 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:09:41.293432 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:09:41.293393 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:09:50.593358 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:09:50.593321 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:10:04.491248 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:10:04.491195 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:10:13.490669 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:10:13.490638 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:10:22.406310 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:10:22.406274 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:10:33.304903 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:10:33.304688 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:10:42.903911 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:10:42.903876 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:10:49.194936 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:10:49.194898 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:11:05.703003 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:05.702961 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:11:16.995087 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:16.995047 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:11:25.901815 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:25.901778 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:11:38.506622 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:38.506581 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:11:46.895233 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:46.895197 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:11:55.894605 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:55.894569 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:11:56.663118 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.663085 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms"] Jul 07 00:11:56.663544 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.663530 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a50d927-75ad-49c3-a48a-f72ff3f6027f" containerName="cleanup" Jul 07 00:11:56.663608 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.663547 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a50d927-75ad-49c3-a48a-f72ff3f6027f" containerName="cleanup" Jul 07 00:11:56.663608 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.663562 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a41f65f-ca12-4745-8d9f-0c86e5e081cb" containerName="image-pruner" Jul 07 00:11:56.663608 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.663568 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a41f65f-ca12-4745-8d9f-0c86e5e081cb" containerName="image-pruner" Jul 07 00:11:56.663710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.663654 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a41f65f-ca12-4745-8d9f-0c86e5e081cb" containerName="image-pruner" Jul 07 00:11:56.663710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.663661 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a50d927-75ad-49c3-a48a-f72ff3f6027f" containerName="cleanup" Jul 07 00:11:56.666824 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.666807 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.668999 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.668975 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-980e0fb5-openshift-default-dockercfg-dzv42\"" Jul 07 00:11:56.689115 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.689093 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms"] Jul 07 00:11:56.777127 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.777093 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-envoy\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.777127 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.777127 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-socket\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.777333 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.777171 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istiod-ca-cert\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.777333 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.777234 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-podinfo\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.777333 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.777283 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-credential-socket\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.777333 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.777331 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-certs\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.777464 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.777346 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-data\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.777464 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.777360 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-token\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.777464 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.777403 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq5tl\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-kube-api-access-nq5tl\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.878336 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878288 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-certs\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.878336 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878334 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-data\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.878336 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-token\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.878686 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878370 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq5tl\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-kube-api-access-nq5tl\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.878686 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878399 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-envoy\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.878686 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878418 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-socket\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.878686 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istiod-ca-cert\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.878686 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878625 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-podinfo\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.878686 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878686 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-credential-socket\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.879020 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878766 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-certs\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.879020 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878904 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-socket\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.879020 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.878984 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-data\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.879186 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.879033 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-credential-socket\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.879186 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.879176 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istiod-ca-cert\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.880569 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.880547 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-envoy\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.881023 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.881006 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-podinfo\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.885715 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.885694 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-token\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.885838 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.885827 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq5tl\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-kube-api-access-nq5tl\") pod \"e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:56.978076 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:56.977987 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:57.118043 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:57.118013 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms"] Jul 07 00:11:57.119155 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:11:57.119130 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a24de3_a49c_4b0c_b337_3ea1f0b4c00a.slice/crio-5dad68f42345d7c795cfe374af0c1107785b73f99e7378f9b0bce3b9e6340640 WatchSource:0}: Error finding container 5dad68f42345d7c795cfe374af0c1107785b73f99e7378f9b0bce3b9e6340640: Status 404 returned error can't find the container with id 5dad68f42345d7c795cfe374af0c1107785b73f99e7378f9b0bce3b9e6340640 Jul 07 00:11:57.121559 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:57.121523 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:11:57.121636 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:57.121615 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:11:57.121711 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:57.121694 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:11:57.138450 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:57.138422 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" event={"ID":"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a","Type":"ContainerStarted","Data":"5dad68f42345d7c795cfe374af0c1107785b73f99e7378f9b0bce3b9e6340640"} Jul 07 00:11:58.146466 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:58.146422 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" event={"ID":"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a","Type":"ContainerStarted","Data":"51550bdec8e624a4490b64bd13691ea74f97157666fb4ec4c4efb73d955375b3"} Jul 07 00:11:58.171331 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:58.171279 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" podStartSLOduration=2.171264172 podStartE2EDuration="2.171264172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:11:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:11:58.169926358 +0000 UTC m=+1428.432598565" watchObservedRunningTime="2026-07-07 00:11:58.171264172 +0000 UTC m=+1428.433936367" Jul 07 00:11:58.978280 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:58.978245 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:58.983285 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:58.983260 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:59.152443 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:59.152411 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:11:59.153406 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:11:59.153388 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:12:03.231807 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:03.231728 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms"] Jul 07 00:12:03.232240 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:03.231907 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" podUID="73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" containerName="istio-proxy" containerID="cri-o://51550bdec8e624a4490b64bd13691ea74f97157666fb4ec4c4efb73d955375b3" gracePeriod=30 Jul 07 00:12:08.215963 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.215932 2583 generic.go:356] "Generic (PLEG): container finished" podID="73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" containerID="51550bdec8e624a4490b64bd13691ea74f97157666fb4ec4c4efb73d955375b3" exitCode=0 Jul 07 00:12:08.216315 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.215980 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" event={"ID":"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a","Type":"ContainerDied","Data":"51550bdec8e624a4490b64bd13691ea74f97157666fb4ec4c4efb73d955375b3"} Jul 07 00:12:08.288452 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.288401 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:12:08.399467 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399372 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-data\") pod \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " Jul 07 00:12:08.399467 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399437 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-token\") pod \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " Jul 07 00:12:08.399710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399512 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-socket\") pod \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " Jul 07 00:12:08.399710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399537 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-podinfo\") pod \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " Jul 07 00:12:08.399710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399558 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-envoy\") pod \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " Jul 07 00:12:08.399710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399577 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-certs\") pod \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " Jul 07 00:12:08.399710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399665 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-credential-socket\") pod \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " Jul 07 00:12:08.399710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399708 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istiod-ca-cert\") pod \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " Jul 07 00:12:08.400022 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399739 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq5tl\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-kube-api-access-nq5tl\") pod \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\" (UID: \"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a\") " Jul 07 00:12:08.400022 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.399909 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" (UID: "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:08.400194 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.400030 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" (UID: "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:08.400194 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.400134 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" (UID: "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:08.400194 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.400073 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:08.400348 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.400231 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" (UID: "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:12:08.400348 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.400278 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-data" (OuterVolumeSpecName: "istio-data") pod "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" (UID: "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:08.401845 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.401819 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" (UID: "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:12:08.401966 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.401868 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" (UID: "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:08.401966 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.401891 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-token" (OuterVolumeSpecName: "istio-token") pod "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" (UID: "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:12:08.402064 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.401970 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-kube-api-access-nq5tl" (OuterVolumeSpecName: "kube-api-access-nq5tl") pod "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" (UID: "73a24de3-a49c-4b0c-b337-3ea1f0b4c00a"). InnerVolumeSpecName "kube-api-access-nq5tl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:12:08.501317 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.501280 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:08.501317 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.501309 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:08.501317 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.501321 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq5tl\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-kube-api-access-nq5tl\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:08.501616 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.501331 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:08.501616 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.501339 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:08.501616 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.501347 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:08.501616 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.501354 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:08.501616 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:08.501362 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:09.223198 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.223171 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" Jul 07 00:12:09.223198 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.223182 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms" event={"ID":"73a24de3-a49c-4b0c-b337-3ea1f0b4c00a","Type":"ContainerDied","Data":"5dad68f42345d7c795cfe374af0c1107785b73f99e7378f9b0bce3b9e6340640"} Jul 07 00:12:09.223680 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.223240 2583 scope.go:117] "RemoveContainer" containerID="51550bdec8e624a4490b64bd13691ea74f97157666fb4ec4c4efb73d955375b3" Jul 07 00:12:09.246257 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.246232 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms"] Jul 07 00:12:09.250110 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.250084 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q"] Jul 07 00:12:09.250715 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.250694 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" containerName="istio-proxy" Jul 07 00:12:09.250715 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.250716 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" containerName="istio-proxy" Jul 07 00:12:09.250887 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.250845 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" containerName="istio-proxy" Jul 07 00:12:09.254090 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.254071 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.256556 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.256191 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4fbb0c76-openshift-default-dockercfg-44wzz\"" Jul 07 00:12:09.262397 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.262346 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-980e0fb5-openshift-default-57f8dd9f85-2kxms"] Jul 07 00:12:09.268748 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.268725 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q"] Jul 07 00:12:09.409379 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.409346 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istiod-ca-cert\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.409595 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.409386 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8d8f\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-kube-api-access-q8d8f\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.409595 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.409431 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-podinfo\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.409595 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.409556 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-socket\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.409595 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.409586 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-envoy\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.409812 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.409624 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-certs\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.409812 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.409659 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-data\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.409812 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.409698 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-credential-socket\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.409812 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.409744 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-token\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.510518 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.510462 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-token\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.510745 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.510520 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istiod-ca-cert\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.510745 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.510559 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8d8f\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-kube-api-access-q8d8f\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.510745 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.510590 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-podinfo\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.510745 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.510650 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-socket\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.510745 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.510668 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-envoy\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.510745 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.510704 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-certs\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.510745 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.510739 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-data\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.511089 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.510784 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-credential-socket\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.511173 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.511140 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-socket\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.511240 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.511195 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-certs\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.511297 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.511232 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-credential-socket\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.511352 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.511335 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-data\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.511406 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.511345 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istiod-ca-cert\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.512946 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.512917 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-envoy\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.513198 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.513179 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-podinfo\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.518092 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.518061 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-token\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.518194 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.518183 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8d8f\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-kube-api-access-q8d8f\") pod \"e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.566184 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.566141 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:09.706188 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.706160 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q"] Jul 07 00:12:09.707772 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:12:09.707746 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5015a234_fcf0_4d7b_8fcd_f9228a89992b.slice/crio-bc2a11b7f2f49e1ddf697ac917b8d2e13fc7f1661c0a2f4678114088ce51f675 WatchSource:0}: Error finding container bc2a11b7f2f49e1ddf697ac917b8d2e13fc7f1661c0a2f4678114088ce51f675: Status 404 returned error can't find the container with id bc2a11b7f2f49e1ddf697ac917b8d2e13fc7f1661c0a2f4678114088ce51f675 Jul 07 00:12:09.709859 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.709827 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:12:09.709927 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.709893 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:12:09.709972 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:09.709926 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:12:10.182152 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:10.182053 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73a24de3-a49c-4b0c-b337-3ea1f0b4c00a" path="/var/lib/kubelet/pods/73a24de3-a49c-4b0c-b337-3ea1f0b4c00a/volumes" Jul 07 00:12:10.231913 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:10.231876 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" event={"ID":"5015a234-fcf0-4d7b-8fcd-f9228a89992b","Type":"ContainerStarted","Data":"5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87"} Jul 07 00:12:10.232371 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:10.231940 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" event={"ID":"5015a234-fcf0-4d7b-8fcd-f9228a89992b","Type":"ContainerStarted","Data":"bc2a11b7f2f49e1ddf697ac917b8d2e13fc7f1661c0a2f4678114088ce51f675"} Jul 07 00:12:10.252483 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:10.252418 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" podStartSLOduration=1.252402446 podStartE2EDuration="1.252402446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:12:10.250551405 +0000 UTC m=+1440.513223599" watchObservedRunningTime="2026-07-07 00:12:10.252402446 +0000 UTC m=+1440.515074640" Jul 07 00:12:10.571504 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:10.571453 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:10.577258 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:10.577232 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:11.239499 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:11.239440 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:11.240431 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:11.240408 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:13.000432 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:13.000395 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:12:21.304845 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:21.304809 2583 generic.go:356] "Generic (PLEG): container finished" podID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" containerID="5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87" exitCode=0 Jul 07 00:12:21.305264 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:21.304870 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" event={"ID":"5015a234-fcf0-4d7b-8fcd-f9228a89992b","Type":"ContainerDied","Data":"5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87"} Jul 07 00:12:21.305264 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:21.305257 2583 scope.go:117] "RemoveContainer" containerID="5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87" Jul 07 00:12:21.306374 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:21.306344 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:12:21.306449 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:21.306432 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:12:21.306531 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:21.306513 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:12:21.790187 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:21.790151 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q"] Jul 07 00:12:22.312212 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:22.312177 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" event={"ID":"5015a234-fcf0-4d7b-8fcd-f9228a89992b","Type":"ContainerStarted","Data":"b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07"} Jul 07 00:12:22.312660 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:22.312234 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" podUID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" containerName="istio-proxy" containerID="cri-o://b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07" gracePeriod=30 Jul 07 00:12:22.496452 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:22.496412 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:12:26.239726 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:26.239690 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:27.570710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.570684 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:27.581542 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581460 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istiod-ca-cert\") pod \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " Jul 07 00:12:27.581542 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581517 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-token\") pod \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " Jul 07 00:12:27.581542 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581542 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-podinfo\") pod \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " Jul 07 00:12:27.581775 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581646 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-data\") pod \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " Jul 07 00:12:27.581775 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581678 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-socket\") pod \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " Jul 07 00:12:27.581775 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581760 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-envoy\") pod \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " Jul 07 00:12:27.581933 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581782 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8d8f\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-kube-api-access-q8d8f\") pod \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " Jul 07 00:12:27.581933 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581820 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-certs\") pod \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " Jul 07 00:12:27.581933 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581851 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-credential-socket\") pod \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\" (UID: \"5015a234-fcf0-4d7b-8fcd-f9228a89992b\") " Jul 07 00:12:27.582101 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581927 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5015a234-fcf0-4d7b-8fcd-f9228a89992b" (UID: "5015a234-fcf0-4d7b-8fcd-f9228a89992b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:12:27.582101 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.581952 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5015a234-fcf0-4d7b-8fcd-f9228a89992b" (UID: "5015a234-fcf0-4d7b-8fcd-f9228a89992b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:27.582101 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.582095 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5015a234-fcf0-4d7b-8fcd-f9228a89992b" (UID: "5015a234-fcf0-4d7b-8fcd-f9228a89992b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:27.582260 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.582143 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5015a234-fcf0-4d7b-8fcd-f9228a89992b" (UID: "5015a234-fcf0-4d7b-8fcd-f9228a89992b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:27.582260 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.582188 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:27.582260 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.582206 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:27.582260 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.582219 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:27.582571 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.582542 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-data" (OuterVolumeSpecName: "istio-data") pod "5015a234-fcf0-4d7b-8fcd-f9228a89992b" (UID: "5015a234-fcf0-4d7b-8fcd-f9228a89992b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:27.583738 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.583711 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5015a234-fcf0-4d7b-8fcd-f9228a89992b" (UID: "5015a234-fcf0-4d7b-8fcd-f9228a89992b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:12:27.583738 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.583730 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-token" (OuterVolumeSpecName: "istio-token") pod "5015a234-fcf0-4d7b-8fcd-f9228a89992b" (UID: "5015a234-fcf0-4d7b-8fcd-f9228a89992b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:12:27.583920 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.583883 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-kube-api-access-q8d8f" (OuterVolumeSpecName: "kube-api-access-q8d8f") pod "5015a234-fcf0-4d7b-8fcd-f9228a89992b" (UID: "5015a234-fcf0-4d7b-8fcd-f9228a89992b"). InnerVolumeSpecName "kube-api-access-q8d8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:12:27.584046 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.584020 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5015a234-fcf0-4d7b-8fcd-f9228a89992b" (UID: "5015a234-fcf0-4d7b-8fcd-f9228a89992b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:27.682859 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.682825 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:27.682859 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.682855 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8d8f\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-kube-api-access-q8d8f\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:27.682859 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.682865 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:27.683093 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.682874 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:27.683093 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.682882 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:27.683093 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.682892 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5015a234-fcf0-4d7b-8fcd-f9228a89992b-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:27.913278 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.913197 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt"] Jul 07 00:12:27.913663 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.913649 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" containerName="istio-proxy" Jul 07 00:12:27.913717 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.913665 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" containerName="istio-proxy" Jul 07 00:12:27.913717 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.913676 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" containerName="istio-proxy" Jul 07 00:12:27.913717 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.913682 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" containerName="istio-proxy" Jul 07 00:12:27.913815 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.913758 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" containerName="istio-proxy" Jul 07 00:12:27.915926 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.915910 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:27.917969 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.917949 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-5ddc79bd-openshift-default-dockercfg-qb8zt\"" Jul 07 00:12:27.933668 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.933646 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt"] Jul 07 00:12:27.985880 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.985844 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-token\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:27.985880 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.985883 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-podinfo\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:27.986142 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.985905 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-credential-socket\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:27.986142 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.985980 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-socket\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:27.986142 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.986043 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3156519-4396-49ea-bf1d-0b75383c7fe3-istiod-ca-cert\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:27.986142 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.986084 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-data\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:27.986142 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.986120 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58dqr\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-kube-api-access-58dqr\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:27.986320 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.986155 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-certs\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:27.986320 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:27.986175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-envoy\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087432 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087396 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-socket\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087656 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087444 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3156519-4396-49ea-bf1d-0b75383c7fe3-istiod-ca-cert\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087656 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087490 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-data\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087656 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087521 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58dqr\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-kube-api-access-58dqr\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087656 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087557 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-certs\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087656 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087587 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-envoy\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087656 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087652 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-token\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087984 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087694 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-podinfo\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087984 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087722 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-credential-socket\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.087984 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087896 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-socket\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.088148 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.087983 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-data\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.088148 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.088010 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-certs\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.088148 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.088095 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-credential-socket\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.088311 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.088296 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3156519-4396-49ea-bf1d-0b75383c7fe3-istiod-ca-cert\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.090121 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.090092 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-envoy\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.090241 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.090222 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-podinfo\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.094617 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.094592 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-token\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.094978 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.094960 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58dqr\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-kube-api-access-58dqr\") pod \"e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.227098 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.227067 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:28.352101 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.352069 2583 generic.go:356] "Generic (PLEG): container finished" podID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" containerID="b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07" exitCode=0 Jul 07 00:12:28.352280 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.352129 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" Jul 07 00:12:28.352280 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.352134 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" event={"ID":"5015a234-fcf0-4d7b-8fcd-f9228a89992b","Type":"ContainerDied","Data":"b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07"} Jul 07 00:12:28.352280 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.352259 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q" event={"ID":"5015a234-fcf0-4d7b-8fcd-f9228a89992b","Type":"ContainerDied","Data":"bc2a11b7f2f49e1ddf697ac917b8d2e13fc7f1661c0a2f4678114088ce51f675"} Jul 07 00:12:28.352422 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.352283 2583 scope.go:117] "RemoveContainer" containerID="b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07" Jul 07 00:12:28.368290 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.368265 2583 scope.go:117] "RemoveContainer" containerID="5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87" Jul 07 00:12:28.370094 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.370072 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt"] Jul 07 00:12:28.371668 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:12:28.371645 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3156519_4396_49ea_bf1d_0b75383c7fe3.slice/crio-07bc6ba168550d0a1a11fa657c9528caa7968f23f238a2af518f410adf238319 WatchSource:0}: Error finding container 07bc6ba168550d0a1a11fa657c9528caa7968f23f238a2af518f410adf238319: Status 404 returned error can't find the container with id 07bc6ba168550d0a1a11fa657c9528caa7968f23f238a2af518f410adf238319 Jul 07 00:12:28.374008 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.373979 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:12:28.374106 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.374052 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:12:28.374106 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.374097 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:12:28.374221 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.374144 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q"] Jul 07 00:12:28.379057 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.379035 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4fbb0c76-openshift-default-5f55cb76b7-jzf5q"] Jul 07 00:12:28.386639 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.386623 2583 scope.go:117] "RemoveContainer" containerID="b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07" Jul 07 00:12:28.386890 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:12:28.386870 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07\": container with ID starting with b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07 not found: ID does not exist" containerID="b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07" Jul 07 00:12:28.386951 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.386896 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07"} err="failed to get container status \"b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07\": rpc error: code = NotFound desc = could not find container \"b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07\": container with ID starting with b74d573a1777d7688d29737089fb88b2d71a1b78d81c9f55c1b66d7993f3ad07 not found: ID does not exist" Jul 07 00:12:28.386951 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.386911 2583 scope.go:117] "RemoveContainer" containerID="5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87" Jul 07 00:12:28.387187 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:12:28.387170 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87\": container with ID starting with 5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87 not found: ID does not exist" containerID="5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87" Jul 07 00:12:28.387235 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:28.387196 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87"} err="failed to get container status \"5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87\": rpc error: code = NotFound desc = could not find container \"5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87\": container with ID starting with 5ef3055e9bead158a250bb839cb889cbc30ad143bfc7e17a54c3ed54b50bef87 not found: ID does not exist" Jul 07 00:12:29.359030 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:29.358997 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" event={"ID":"e3156519-4396-49ea-bf1d-0b75383c7fe3","Type":"ContainerStarted","Data":"e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149"} Jul 07 00:12:29.359529 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:29.359059 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" event={"ID":"e3156519-4396-49ea-bf1d-0b75383c7fe3","Type":"ContainerStarted","Data":"07bc6ba168550d0a1a11fa657c9528caa7968f23f238a2af518f410adf238319"} Jul 07 00:12:29.386741 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:29.386693 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" podStartSLOduration=2.386676427 podStartE2EDuration="2.386676427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:12:29.382684716 +0000 UTC m=+1459.645356910" watchObservedRunningTime="2026-07-07 00:12:29.386676427 +0000 UTC m=+1459.649348622" Jul 07 00:12:30.181369 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:30.181335 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" path="/var/lib/kubelet/pods/5015a234-fcf0-4d7b-8fcd-f9228a89992b/volumes" Jul 07 00:12:30.227500 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:30.227405 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:30.232253 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:30.232227 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:30.366445 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:30.366418 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:30.367392 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:30.367376 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:33.640497 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:33.640427 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt"] Jul 07 00:12:33.640974 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:33.640669 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" podUID="e3156519-4396-49ea-bf1d-0b75383c7fe3" containerName="istio-proxy" containerID="cri-o://e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149" gracePeriod=30 Jul 07 00:12:38.886353 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.886328 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:38.981647 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981569 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-token\") pod \"e3156519-4396-49ea-bf1d-0b75383c7fe3\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " Jul 07 00:12:38.981647 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981608 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-credential-socket\") pod \"e3156519-4396-49ea-bf1d-0b75383c7fe3\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " Jul 07 00:12:38.981647 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981627 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-socket\") pod \"e3156519-4396-49ea-bf1d-0b75383c7fe3\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " Jul 07 00:12:38.981647 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981641 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-data\") pod \"e3156519-4396-49ea-bf1d-0b75383c7fe3\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " Jul 07 00:12:38.982036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981694 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3156519-4396-49ea-bf1d-0b75383c7fe3-istiod-ca-cert\") pod \"e3156519-4396-49ea-bf1d-0b75383c7fe3\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " Jul 07 00:12:38.982036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981757 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58dqr\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-kube-api-access-58dqr\") pod \"e3156519-4396-49ea-bf1d-0b75383c7fe3\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " Jul 07 00:12:38.982036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981793 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-podinfo\") pod \"e3156519-4396-49ea-bf1d-0b75383c7fe3\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " Jul 07 00:12:38.982036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981881 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-certs\") pod \"e3156519-4396-49ea-bf1d-0b75383c7fe3\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " Jul 07 00:12:38.982036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981912 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-envoy\") pod \"e3156519-4396-49ea-bf1d-0b75383c7fe3\" (UID: \"e3156519-4396-49ea-bf1d-0b75383c7fe3\") " Jul 07 00:12:38.982036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981902 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e3156519-4396-49ea-bf1d-0b75383c7fe3" (UID: "e3156519-4396-49ea-bf1d-0b75383c7fe3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:38.982036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981958 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e3156519-4396-49ea-bf1d-0b75383c7fe3" (UID: "e3156519-4396-49ea-bf1d-0b75383c7fe3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:38.982036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.981981 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-data" (OuterVolumeSpecName: "istio-data") pod "e3156519-4396-49ea-bf1d-0b75383c7fe3" (UID: "e3156519-4396-49ea-bf1d-0b75383c7fe3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:38.982440 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.982135 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e3156519-4396-49ea-bf1d-0b75383c7fe3" (UID: "e3156519-4396-49ea-bf1d-0b75383c7fe3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:38.982440 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.982235 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3156519-4396-49ea-bf1d-0b75383c7fe3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e3156519-4396-49ea-bf1d-0b75383c7fe3" (UID: "e3156519-4396-49ea-bf1d-0b75383c7fe3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:12:38.982440 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.982363 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:38.982440 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.982381 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:38.982440 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.982395 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:38.982440 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.982410 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:38.982440 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.982423 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e3156519-4396-49ea-bf1d-0b75383c7fe3-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:38.983868 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.983835 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-token" (OuterVolumeSpecName: "istio-token") pod "e3156519-4396-49ea-bf1d-0b75383c7fe3" (UID: "e3156519-4396-49ea-bf1d-0b75383c7fe3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:12:38.984001 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.983903 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-kube-api-access-58dqr" (OuterVolumeSpecName: "kube-api-access-58dqr") pod "e3156519-4396-49ea-bf1d-0b75383c7fe3" (UID: "e3156519-4396-49ea-bf1d-0b75383c7fe3"). InnerVolumeSpecName "kube-api-access-58dqr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:12:38.984001 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.983907 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e3156519-4396-49ea-bf1d-0b75383c7fe3" (UID: "e3156519-4396-49ea-bf1d-0b75383c7fe3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:12:38.984096 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:38.984002 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e3156519-4396-49ea-bf1d-0b75383c7fe3" (UID: "e3156519-4396-49ea-bf1d-0b75383c7fe3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:12:39.082945 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.082905 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:39.082945 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.082939 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58dqr\" (UniqueName: \"kubernetes.io/projected/e3156519-4396-49ea-bf1d-0b75383c7fe3-kube-api-access-58dqr\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:39.082945 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.082953 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:39.083184 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.082966 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e3156519-4396-49ea-bf1d-0b75383c7fe3-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:12:39.419244 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.419209 2583 generic.go:356] "Generic (PLEG): container finished" podID="e3156519-4396-49ea-bf1d-0b75383c7fe3" containerID="e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149" exitCode=0 Jul 07 00:12:39.419422 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.419250 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" Jul 07 00:12:39.419422 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.419291 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" event={"ID":"e3156519-4396-49ea-bf1d-0b75383c7fe3","Type":"ContainerDied","Data":"e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149"} Jul 07 00:12:39.419422 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.419348 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt" event={"ID":"e3156519-4396-49ea-bf1d-0b75383c7fe3","Type":"ContainerDied","Data":"07bc6ba168550d0a1a11fa657c9528caa7968f23f238a2af518f410adf238319"} Jul 07 00:12:39.419422 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.419374 2583 scope.go:117] "RemoveContainer" containerID="e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149" Jul 07 00:12:39.429565 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.429547 2583 scope.go:117] "RemoveContainer" containerID="e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149" Jul 07 00:12:39.429807 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:12:39.429789 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149\": container with ID starting with e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149 not found: ID does not exist" containerID="e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149" Jul 07 00:12:39.429868 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.429818 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149"} err="failed to get container status \"e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149\": rpc error: code = NotFound desc = could not find container \"e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149\": container with ID starting with e99256f2e48e47db803dfaa8550e83ebc7eb17f98448a6de1517d5ab2d8a4149 not found: ID does not exist" Jul 07 00:12:39.440741 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.440707 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt"] Jul 07 00:12:39.443736 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:39.443714 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-5ddc79bd-openshift-default-75cd4b564c-db8kt"] Jul 07 00:12:40.183113 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:12:40.183080 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3156519-4396-49ea-bf1d-0b75383c7fe3" path="/var/lib/kubelet/pods/e3156519-4396-49ea-bf1d-0b75383c7fe3/volumes" Jul 07 00:14:02.403487 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:02.403392 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:14:09.998273 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:09.998237 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:14:15.707296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.707263 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd"] Jul 07 00:14:15.707759 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.707697 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3156519-4396-49ea-bf1d-0b75383c7fe3" containerName="istio-proxy" Jul 07 00:14:15.707759 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.707710 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3156519-4396-49ea-bf1d-0b75383c7fe3" containerName="istio-proxy" Jul 07 00:14:15.707831 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.707788 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5015a234-fcf0-4d7b-8fcd-f9228a89992b" containerName="istio-proxy" Jul 07 00:14:15.707831 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.707798 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3156519-4396-49ea-bf1d-0b75383c7fe3" containerName="istio-proxy" Jul 07 00:14:15.710778 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.710762 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.713079 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.713058 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4de63527-openshift-default-dockercfg-xh5db\"" Jul 07 00:14:15.734444 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.734422 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd"] Jul 07 00:14:15.752740 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.752714 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-envoy\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.752873 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.752745 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-credential-socket\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.752873 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.752772 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-socket\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.752873 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.752798 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-podinfo\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.752873 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.752843 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-certs\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.753031 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.752898 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-token\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.753031 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.752925 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9bw8\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-kube-api-access-w9bw8\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.753031 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.752953 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e1a08a-a178-4f17-a992-c010c98adb5a-istiod-ca-cert\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.753031 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.753008 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-data\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.854436 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854405 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-socket\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.854651 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854445 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-podinfo\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.854651 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854499 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-certs\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.854651 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854532 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-token\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.854651 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854552 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9bw8\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-kube-api-access-w9bw8\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.854651 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e1a08a-a178-4f17-a992-c010c98adb5a-istiod-ca-cert\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.854651 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854628 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-data\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.855000 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854683 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-envoy\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.855000 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854711 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-credential-socket\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.855000 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854860 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-socket\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.855000 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.854901 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-certs\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.855218 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.855076 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-data\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.855218 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.855179 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-credential-socket\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.855311 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.855289 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e1a08a-a178-4f17-a992-c010c98adb5a-istiod-ca-cert\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.856883 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.856856 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-envoy\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.857111 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.857094 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-podinfo\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.861738 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.861713 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-token\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:15.862418 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:15.862394 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9bw8\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-kube-api-access-w9bw8\") pod \"e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:16.020640 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:16.020610 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:16.192803 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:16.192782 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd"] Jul 07 00:14:16.194501 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:14:16.194452 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e1a08a_a178_4f17_a992_c010c98adb5a.slice/crio-2b7b56f92986a93792120fb132081b6ac9941117c13dc6dc7ba9f3830695ac0e WatchSource:0}: Error finding container 2b7b56f92986a93792120fb132081b6ac9941117c13dc6dc7ba9f3830695ac0e: Status 404 returned error can't find the container with id 2b7b56f92986a93792120fb132081b6ac9941117c13dc6dc7ba9f3830695ac0e Jul 07 00:14:16.196448 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:16.196415 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:14:16.196580 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:16.196509 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:14:16.196580 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:16.196556 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:14:16.982262 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:16.982220 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" event={"ID":"06e1a08a-a178-4f17-a992-c010c98adb5a","Type":"ContainerStarted","Data":"7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a"} Jul 07 00:14:16.982657 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:16.982269 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" event={"ID":"06e1a08a-a178-4f17-a992-c010c98adb5a","Type":"ContainerStarted","Data":"2b7b56f92986a93792120fb132081b6ac9941117c13dc6dc7ba9f3830695ac0e"} Jul 07 00:14:17.010869 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:17.010818 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" podStartSLOduration=2.010803056 podStartE2EDuration="2.010803056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:14:17.008655087 +0000 UTC m=+1567.271327283" watchObservedRunningTime="2026-07-07 00:14:17.010803056 +0000 UTC m=+1567.273475242" Jul 07 00:14:17.021028 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:17.020998 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:17.026184 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:17.026159 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:17.988086 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:17.988053 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:17.989141 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:17.989121 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:18.802649 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:18.802609 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:14:28.604724 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:28.604687 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:14:32.989694 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:32.989649 2583 patch_prober.go:28] interesting pod/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 07 00:14:32.990205 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:32.989733 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" podUID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 07 00:14:33.097943 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:33.097907 2583 generic.go:356] "Generic (PLEG): container finished" podID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerID="7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a" exitCode=0 Jul 07 00:14:33.098116 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:33.097978 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" event={"ID":"06e1a08a-a178-4f17-a992-c010c98adb5a","Type":"ContainerDied","Data":"7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a"} Jul 07 00:14:33.098389 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:33.098376 2583 scope.go:117] "RemoveContainer" containerID="7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a" Jul 07 00:14:33.099466 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:33.099432 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:14:33.099574 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:33.099546 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:14:33.099614 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:33.099603 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:14:34.105931 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:34.105891 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" event={"ID":"06e1a08a-a178-4f17-a992-c010c98adb5a","Type":"ContainerStarted","Data":"5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d"} Jul 07 00:14:35.021806 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:35.021766 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:35.025645 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:35.025620 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:35.111849 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:35.111815 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:35.112972 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:35.112953 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:36.945511 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:36.945460 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd"] Jul 07 00:14:37.123256 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:37.123221 2583 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" secret="" err="secret \"e2e-mt-4de63527-openshift-default-dockercfg-xh5db\" not found" Jul 07 00:14:38.128649 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:38.128592 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" podUID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerName="istio-proxy" containerID="cri-o://5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d" gracePeriod=30 Jul 07 00:14:38.690497 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:38.690438 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:14:43.386369 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.386334 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:43.425613 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425529 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-certs\") pod \"06e1a08a-a178-4f17-a992-c010c98adb5a\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " Jul 07 00:14:43.425613 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425596 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-credential-socket\") pod \"06e1a08a-a178-4f17-a992-c010c98adb5a\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " Jul 07 00:14:43.425817 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425620 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-envoy\") pod \"06e1a08a-a178-4f17-a992-c010c98adb5a\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " Jul 07 00:14:43.425817 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425647 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-socket\") pod \"06e1a08a-a178-4f17-a992-c010c98adb5a\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " Jul 07 00:14:43.425817 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425665 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9bw8\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-kube-api-access-w9bw8\") pod \"06e1a08a-a178-4f17-a992-c010c98adb5a\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " Jul 07 00:14:43.425817 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425686 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-data\") pod \"06e1a08a-a178-4f17-a992-c010c98adb5a\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " Jul 07 00:14:43.425817 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425707 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-podinfo\") pod \"06e1a08a-a178-4f17-a992-c010c98adb5a\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " Jul 07 00:14:43.425817 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425736 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-token\") pod \"06e1a08a-a178-4f17-a992-c010c98adb5a\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " Jul 07 00:14:43.426151 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425809 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "06e1a08a-a178-4f17-a992-c010c98adb5a" (UID: "06e1a08a-a178-4f17-a992-c010c98adb5a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:14:43.426151 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425817 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e1a08a-a178-4f17-a992-c010c98adb5a-istiod-ca-cert\") pod \"06e1a08a-a178-4f17-a992-c010c98adb5a\" (UID: \"06e1a08a-a178-4f17-a992-c010c98adb5a\") " Jul 07 00:14:43.426151 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425880 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "06e1a08a-a178-4f17-a992-c010c98adb5a" (UID: "06e1a08a-a178-4f17-a992-c010c98adb5a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:14:43.426151 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.425987 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "06e1a08a-a178-4f17-a992-c010c98adb5a" (UID: "06e1a08a-a178-4f17-a992-c010c98adb5a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:14:43.426383 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.426173 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:14:43.426383 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.426195 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:14:43.426383 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.426210 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:14:43.426561 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.426444 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06e1a08a-a178-4f17-a992-c010c98adb5a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "06e1a08a-a178-4f17-a992-c010c98adb5a" (UID: "06e1a08a-a178-4f17-a992-c010c98adb5a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:14:43.426848 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.426826 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-data" (OuterVolumeSpecName: "istio-data") pod "06e1a08a-a178-4f17-a992-c010c98adb5a" (UID: "06e1a08a-a178-4f17-a992-c010c98adb5a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:14:43.428245 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.428220 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-token" (OuterVolumeSpecName: "istio-token") pod "06e1a08a-a178-4f17-a992-c010c98adb5a" (UID: "06e1a08a-a178-4f17-a992-c010c98adb5a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:14:43.428336 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.428281 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "06e1a08a-a178-4f17-a992-c010c98adb5a" (UID: "06e1a08a-a178-4f17-a992-c010c98adb5a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:14:43.428525 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.428502 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "06e1a08a-a178-4f17-a992-c010c98adb5a" (UID: "06e1a08a-a178-4f17-a992-c010c98adb5a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:14:43.428642 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.428626 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-kube-api-access-w9bw8" (OuterVolumeSpecName: "kube-api-access-w9bw8") pod "06e1a08a-a178-4f17-a992-c010c98adb5a" (UID: "06e1a08a-a178-4f17-a992-c010c98adb5a"). InnerVolumeSpecName "kube-api-access-w9bw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:14:43.526877 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.526840 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06e1a08a-a178-4f17-a992-c010c98adb5a-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:14:43.526877 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.526871 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:14:43.526877 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.526882 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9bw8\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-kube-api-access-w9bw8\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:14:43.527124 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.526891 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:14:43.527124 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.526902 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:14:43.527124 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:43.526909 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06e1a08a-a178-4f17-a992-c010c98adb5a-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:14:44.165122 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.165086 2583 generic.go:356] "Generic (PLEG): container finished" podID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerID="5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d" exitCode=0 Jul 07 00:14:44.165304 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.165140 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" Jul 07 00:14:44.165304 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.165174 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" event={"ID":"06e1a08a-a178-4f17-a992-c010c98adb5a","Type":"ContainerDied","Data":"5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d"} Jul 07 00:14:44.165304 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.165256 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd" event={"ID":"06e1a08a-a178-4f17-a992-c010c98adb5a","Type":"ContainerDied","Data":"2b7b56f92986a93792120fb132081b6ac9941117c13dc6dc7ba9f3830695ac0e"} Jul 07 00:14:44.165304 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.165281 2583 scope.go:117] "RemoveContainer" containerID="5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d" Jul 07 00:14:44.174909 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.174889 2583 scope.go:117] "RemoveContainer" containerID="7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a" Jul 07 00:14:44.183989 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.183967 2583 scope.go:117] "RemoveContainer" containerID="5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d" Jul 07 00:14:44.184226 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:14:44.184204 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d\": container with ID starting with 5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d not found: ID does not exist" containerID="5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d" Jul 07 00:14:44.184301 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.184235 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d"} err="failed to get container status \"5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d\": rpc error: code = NotFound desc = could not find container \"5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d\": container with ID starting with 5b3205a8647dffa9a598128a8b929ef2ee1f13fcfc14d6a7deea23880d29325d not found: ID does not exist" Jul 07 00:14:44.184301 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.184253 2583 scope.go:117] "RemoveContainer" containerID="7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a" Jul 07 00:14:44.184465 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:14:44.184448 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a\": container with ID starting with 7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a not found: ID does not exist" containerID="7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a" Jul 07 00:14:44.184551 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.184489 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a"} err="failed to get container status \"7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a\": rpc error: code = NotFound desc = could not find container \"7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a\": container with ID starting with 7e48642e41a36c1f7ccb8cd6429ff097d7b5770955c87f44bfb65395f0922d6a not found: ID does not exist" Jul 07 00:14:44.186145 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.186124 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd"] Jul 07 00:14:44.189437 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:44.189418 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4de63527-openshift-default-7f957d4b8c-r94gd"] Jul 07 00:14:46.181587 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:46.181548 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06e1a08a-a178-4f17-a992-c010c98adb5a" path="/var/lib/kubelet/pods/06e1a08a-a178-4f17-a992-c010c98adb5a/volumes" Jul 07 00:14:56.474103 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.474065 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq"] Jul 07 00:14:56.474678 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.474491 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerName="istio-proxy" Jul 07 00:14:56.474678 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.474505 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerName="istio-proxy" Jul 07 00:14:56.474678 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.474531 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerName="istio-proxy" Jul 07 00:14:56.474678 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.474537 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerName="istio-proxy" Jul 07 00:14:56.474678 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.474602 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerName="istio-proxy" Jul 07 00:14:56.477831 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.477806 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.482976 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.482951 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-c75b7974-openshift-default-dockercfg-zlll6\"" Jul 07 00:14:56.494912 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.494884 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq"] Jul 07 00:14:56.551032 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.551000 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-credential-socket\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.551232 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.551108 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-token\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.551232 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.551135 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istiod-ca-cert\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.551232 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.551185 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-certs\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.551232 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.551225 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-data\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.551460 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.551306 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-socket\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.551460 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.551330 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-envoy\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.551460 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.551357 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fptgf\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-kube-api-access-fptgf\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.551460 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.551412 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-podinfo\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652124 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652087 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-socket\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652124 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-envoy\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652373 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652152 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fptgf\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-kube-api-access-fptgf\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652373 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652199 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-podinfo\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652373 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652231 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-credential-socket\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652373 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652277 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-token\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652373 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652298 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istiod-ca-cert\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652373 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-certs\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652753 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652381 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-data\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652753 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652530 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-socket\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652753 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652712 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-credential-socket\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.652916 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.652883 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-certs\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.653079 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.653057 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-data\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.653283 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.653257 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istiod-ca-cert\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.654721 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.654693 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-envoy\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.654919 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.654902 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-podinfo\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.660390 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.660366 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-token\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.660460 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.660375 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fptgf\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-kube-api-access-fptgf\") pod \"e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.788559 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.788523 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:56.954812 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.954784 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq"] Jul 07 00:14:56.957136 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:14:56.957107 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd1fca0_a44a_48e4_9363_4f1dccc89b24.slice/crio-53744da3e50fb1e739324df6a7ba9c2123d25e8c688bbcb30fbbf984ce309293 WatchSource:0}: Error finding container 53744da3e50fb1e739324df6a7ba9c2123d25e8c688bbcb30fbbf984ce309293: Status 404 returned error can't find the container with id 53744da3e50fb1e739324df6a7ba9c2123d25e8c688bbcb30fbbf984ce309293 Jul 07 00:14:56.959341 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.959310 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:14:56.959411 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.959371 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:14:56.959411 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:56.959401 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:14:57.246022 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:57.245991 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" event={"ID":"8dd1fca0-a44a-48e4-9363-4f1dccc89b24","Type":"ContainerStarted","Data":"7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2"} Jul 07 00:14:57.246204 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:57.246050 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" event={"ID":"8dd1fca0-a44a-48e4-9363-4f1dccc89b24","Type":"ContainerStarted","Data":"53744da3e50fb1e739324df6a7ba9c2123d25e8c688bbcb30fbbf984ce309293"} Jul 07 00:14:57.267328 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:57.267279 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" podStartSLOduration=1.2672621689999999 podStartE2EDuration="1.267262169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:14:57.264828974 +0000 UTC m=+1607.527501170" watchObservedRunningTime="2026-07-07 00:14:57.267262169 +0000 UTC m=+1607.529934363" Jul 07 00:14:57.789293 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:57.789260 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:58.794554 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:58.794520 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:59.258971 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:59.258944 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:14:59.260173 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:14:59.260150 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:15:00.210744 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.210713 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s"] Jul 07 00:15:00.211379 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.211359 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="06e1a08a-a178-4f17-a992-c010c98adb5a" containerName="istio-proxy" Jul 07 00:15:00.213399 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.213378 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" Jul 07 00:15:00.216840 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.216818 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-7qrr6\"" Jul 07 00:15:00.225027 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.225007 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29723055-mmnms"] Jul 07 00:15:00.227196 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.227181 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" Jul 07 00:15:00.231681 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.231659 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s"] Jul 07 00:15:00.253677 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.253647 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723055-mmnms"] Jul 07 00:15:00.285921 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.285890 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dj2pw\" (UniqueName: \"kubernetes.io/projected/fcf64787-d325-4b03-a982-3dcb538b0d5e-kube-api-access-dj2pw\") pod \"maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s\" (UID: \"fcf64787-d325-4b03-a982-3dcb538b0d5e\") " pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" Jul 07 00:15:00.286083 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.285945 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg42c\" (UniqueName: \"kubernetes.io/projected/54cc54a8-c744-462a-99d3-a9b43a4de909-kube-api-access-mg42c\") pod \"maas-api-key-cleanup-29723055-mmnms\" (UID: \"54cc54a8-c744-462a-99d3-a9b43a4de909\") " pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" Jul 07 00:15:00.387357 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.387312 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dj2pw\" (UniqueName: \"kubernetes.io/projected/fcf64787-d325-4b03-a982-3dcb538b0d5e-kube-api-access-dj2pw\") pod \"maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s\" (UID: \"fcf64787-d325-4b03-a982-3dcb538b0d5e\") " pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" Jul 07 00:15:00.387619 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.387419 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg42c\" (UniqueName: \"kubernetes.io/projected/54cc54a8-c744-462a-99d3-a9b43a4de909-kube-api-access-mg42c\") pod \"maas-api-key-cleanup-29723055-mmnms\" (UID: \"54cc54a8-c744-462a-99d3-a9b43a4de909\") " pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" Jul 07 00:15:00.394988 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.394966 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dj2pw\" (UniqueName: \"kubernetes.io/projected/fcf64787-d325-4b03-a982-3dcb538b0d5e-kube-api-access-dj2pw\") pod \"maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s\" (UID: \"fcf64787-d325-4b03-a982-3dcb538b0d5e\") " pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" Jul 07 00:15:00.395150 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.395022 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg42c\" (UniqueName: \"kubernetes.io/projected/54cc54a8-c744-462a-99d3-a9b43a4de909-kube-api-access-mg42c\") pod \"maas-api-key-cleanup-29723055-mmnms\" (UID: \"54cc54a8-c744-462a-99d3-a9b43a4de909\") " pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" Jul 07 00:15:00.525609 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.525577 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" Jul 07 00:15:00.539698 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.539675 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" Jul 07 00:15:00.683768 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.683739 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s"] Jul 07 00:15:00.684708 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:15:00.684678 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf64787_d325_4b03_a982_3dcb538b0d5e.slice/crio-7cb5a3fed61ababe54fe82c2f03be8c3c5935c1fc325beec8bdb69aeb62f4b56 WatchSource:0}: Error finding container 7cb5a3fed61ababe54fe82c2f03be8c3c5935c1fc325beec8bdb69aeb62f4b56: Status 404 returned error can't find the container with id 7cb5a3fed61ababe54fe82c2f03be8c3c5935c1fc325beec8bdb69aeb62f4b56 Jul 07 00:15:00.912908 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:00.912883 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29723055-mmnms"] Jul 07 00:15:00.914613 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:15:00.914588 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54cc54a8_c744_462a_99d3_a9b43a4de909.slice/crio-10a6c5bcc93bdafaa832c208d45783c7ced70a654447181c57c2cd08ed1a9cad WatchSource:0}: Error finding container 10a6c5bcc93bdafaa832c208d45783c7ced70a654447181c57c2cd08ed1a9cad: Status 404 returned error can't find the container with id 10a6c5bcc93bdafaa832c208d45783c7ced70a654447181c57c2cd08ed1a9cad Jul 07 00:15:01.278342 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:01.278301 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" event={"ID":"fcf64787-d325-4b03-a982-3dcb538b0d5e","Type":"ContainerStarted","Data":"bdbb1653ccb0cdbf6941a79bdf3d3c076dfeed24250b6a0a4de27fd0610ac53a"} Jul 07 00:15:01.278775 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:01.278394 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" event={"ID":"fcf64787-d325-4b03-a982-3dcb538b0d5e","Type":"ContainerStarted","Data":"7cb5a3fed61ababe54fe82c2f03be8c3c5935c1fc325beec8bdb69aeb62f4b56"} Jul 07 00:15:01.279918 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:01.279897 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" event={"ID":"54cc54a8-c744-462a-99d3-a9b43a4de909","Type":"ContainerStarted","Data":"b271580ab130d11348b2a766672e307dbc72a01795e882689745962ca9775e8a"} Jul 07 00:15:01.279987 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:01.279938 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" event={"ID":"54cc54a8-c744-462a-99d3-a9b43a4de909","Type":"ContainerStarted","Data":"10a6c5bcc93bdafaa832c208d45783c7ced70a654447181c57c2cd08ed1a9cad"} Jul 07 00:15:01.294145 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:01.294095 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" podStartSLOduration=1.294081277 podStartE2EDuration="1.294081277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:01.291405125 +0000 UTC m=+1611.554077321" watchObservedRunningTime="2026-07-07 00:15:01.294081277 +0000 UTC m=+1611.556753472" Jul 07 00:15:01.305344 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:01.305304 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" podStartSLOduration=1.305291246 podStartE2EDuration="1.305291246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:15:01.303915629 +0000 UTC m=+1611.566587836" watchObservedRunningTime="2026-07-07 00:15:01.305291246 +0000 UTC m=+1611.567963440" Jul 07 00:15:02.287077 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:02.287040 2583 generic.go:356] "Generic (PLEG): container finished" podID="54cc54a8-c744-462a-99d3-a9b43a4de909" containerID="b271580ab130d11348b2a766672e307dbc72a01795e882689745962ca9775e8a" exitCode=0 Jul 07 00:15:02.287452 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:02.287127 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" event={"ID":"54cc54a8-c744-462a-99d3-a9b43a4de909","Type":"ContainerDied","Data":"b271580ab130d11348b2a766672e307dbc72a01795e882689745962ca9775e8a"} Jul 07 00:15:02.288635 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:02.288615 2583 generic.go:356] "Generic (PLEG): container finished" podID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerID="bdbb1653ccb0cdbf6941a79bdf3d3c076dfeed24250b6a0a4de27fd0610ac53a" exitCode=7 Jul 07 00:15:02.288704 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:02.288688 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" event={"ID":"fcf64787-d325-4b03-a982-3dcb538b0d5e","Type":"ContainerDied","Data":"bdbb1653ccb0cdbf6941a79bdf3d3c076dfeed24250b6a0a4de27fd0610ac53a"} Jul 07 00:15:02.288888 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:02.288876 2583 scope.go:117] "RemoveContainer" containerID="bdbb1653ccb0cdbf6941a79bdf3d3c076dfeed24250b6a0a4de27fd0610ac53a" Jul 07 00:15:03.296382 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:03.296347 2583 generic.go:356] "Generic (PLEG): container finished" podID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerID="05e4f62aa1288f4b0386e45578ff92e0b56f44694c3992211a647740a75a7842" exitCode=7 Jul 07 00:15:03.296918 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:03.296429 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" event={"ID":"fcf64787-d325-4b03-a982-3dcb538b0d5e","Type":"ContainerDied","Data":"05e4f62aa1288f4b0386e45578ff92e0b56f44694c3992211a647740a75a7842"} Jul 07 00:15:03.296918 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:03.296502 2583 scope.go:117] "RemoveContainer" containerID="bdbb1653ccb0cdbf6941a79bdf3d3c076dfeed24250b6a0a4de27fd0610ac53a" Jul 07 00:15:03.296918 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:03.296747 2583 scope.go:117] "RemoveContainer" containerID="05e4f62aa1288f4b0386e45578ff92e0b56f44694c3992211a647740a75a7842" Jul 07 00:15:03.297442 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:15:03.297014 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s_opendatahub(fcf64787-d325-4b03-a982-3dcb538b0d5e)\"" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" Jul 07 00:15:03.436341 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:03.436315 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" Jul 07 00:15:03.518122 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:03.518090 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg42c\" (UniqueName: \"kubernetes.io/projected/54cc54a8-c744-462a-99d3-a9b43a4de909-kube-api-access-mg42c\") pod \"54cc54a8-c744-462a-99d3-a9b43a4de909\" (UID: \"54cc54a8-c744-462a-99d3-a9b43a4de909\") " Jul 07 00:15:03.520111 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:03.520079 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54cc54a8-c744-462a-99d3-a9b43a4de909-kube-api-access-mg42c" (OuterVolumeSpecName: "kube-api-access-mg42c") pod "54cc54a8-c744-462a-99d3-a9b43a4de909" (UID: "54cc54a8-c744-462a-99d3-a9b43a4de909"). InnerVolumeSpecName "kube-api-access-mg42c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:03.619970 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:03.619878 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg42c\" (UniqueName: \"kubernetes.io/projected/54cc54a8-c744-462a-99d3-a9b43a4de909-kube-api-access-mg42c\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:15:04.304532 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:04.304501 2583 scope.go:117] "RemoveContainer" containerID="05e4f62aa1288f4b0386e45578ff92e0b56f44694c3992211a647740a75a7842" Jul 07 00:15:04.304932 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:15:04.304705 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s_opendatahub(fcf64787-d325-4b03-a982-3dcb538b0d5e)\"" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" Jul 07 00:15:04.305663 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:04.305639 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" event={"ID":"54cc54a8-c744-462a-99d3-a9b43a4de909","Type":"ContainerDied","Data":"10a6c5bcc93bdafaa832c208d45783c7ced70a654447181c57c2cd08ed1a9cad"} Jul 07 00:15:04.305663 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:04.305654 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29723055-mmnms" Jul 07 00:15:04.305854 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:04.305678 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10a6c5bcc93bdafaa832c208d45783c7ced70a654447181c57c2cd08ed1a9cad" Jul 07 00:15:05.798874 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:05.798845 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:15:14.260674 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:14.260636 2583 patch_prober.go:28] interesting pod/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 00:15:14.261092 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:14.260686 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 00:15:14.371727 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:14.371691 2583 generic.go:356] "Generic (PLEG): container finished" podID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerID="7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2" exitCode=0 Jul 07 00:15:14.371901 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:14.371763 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" event={"ID":"8dd1fca0-a44a-48e4-9363-4f1dccc89b24","Type":"ContainerDied","Data":"7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2"} Jul 07 00:15:14.372162 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:14.372146 2583 scope.go:117] "RemoveContainer" containerID="7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2" Jul 07 00:15:14.373171 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:14.373144 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:15:14.373236 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:14.373211 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:15:14.373274 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:14.373242 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:15:15.379628 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:15.379585 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" event={"ID":"8dd1fca0-a44a-48e4-9363-4f1dccc89b24","Type":"ContainerStarted","Data":"c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b"} Jul 07 00:15:15.788995 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:15.788959 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:15:15.789220 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:15.789181 2583 patch_prober.go:28] interesting pod/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 00:15:15.789299 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:15.789235 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 00:15:16.176078 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:16.175994 2583 scope.go:117] "RemoveContainer" containerID="05e4f62aa1288f4b0386e45578ff92e0b56f44694c3992211a647740a75a7842" Jul 07 00:15:16.789337 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:16.789304 2583 patch_prober.go:28] interesting pod/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 00:15:16.789792 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:16.789357 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 00:15:17.393202 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:17.393159 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" event={"ID":"fcf64787-d325-4b03-a982-3dcb538b0d5e","Type":"ContainerStarted","Data":"0fbd80bb72af210df6724a1fa4ba5732ac3ce2104670f5e8e1a8ab071e0e89c7"} Jul 07 00:15:17.709446 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:17.709366 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:15:17.789330 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:17.789295 2583 patch_prober.go:28] interesting pod/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 00:15:17.789497 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:17.789352 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 00:15:18.400860 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:18.400825 2583 generic.go:356] "Generic (PLEG): container finished" podID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerID="0fbd80bb72af210df6724a1fa4ba5732ac3ce2104670f5e8e1a8ab071e0e89c7" exitCode=0 Jul 07 00:15:18.401044 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:18.400897 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" event={"ID":"fcf64787-d325-4b03-a982-3dcb538b0d5e","Type":"ContainerDied","Data":"0fbd80bb72af210df6724a1fa4ba5732ac3ce2104670f5e8e1a8ab071e0e89c7"} Jul 07 00:15:18.401044 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:18.400955 2583 scope.go:117] "RemoveContainer" containerID="05e4f62aa1288f4b0386e45578ff92e0b56f44694c3992211a647740a75a7842" Jul 07 00:15:18.443857 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:18.443815 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s"] Jul 07 00:15:18.789214 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:18.789178 2583 patch_prober.go:28] interesting pod/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 07 00:15:18.789399 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:18.789240 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 07 00:15:19.550460 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:19.550431 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" Jul 07 00:15:19.586717 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:19.586692 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dj2pw\" (UniqueName: \"kubernetes.io/projected/fcf64787-d325-4b03-a982-3dcb538b0d5e-kube-api-access-dj2pw\") pod \"fcf64787-d325-4b03-a982-3dcb538b0d5e\" (UID: \"fcf64787-d325-4b03-a982-3dcb538b0d5e\") " Jul 07 00:15:19.588794 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:19.588763 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcf64787-d325-4b03-a982-3dcb538b0d5e-kube-api-access-dj2pw" (OuterVolumeSpecName: "kube-api-access-dj2pw") pod "fcf64787-d325-4b03-a982-3dcb538b0d5e" (UID: "fcf64787-d325-4b03-a982-3dcb538b0d5e"). InnerVolumeSpecName "kube-api-access-dj2pw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:15:19.688046 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:19.687953 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dj2pw\" (UniqueName: \"kubernetes.io/projected/fcf64787-d325-4b03-a982-3dcb538b0d5e-kube-api-access-dj2pw\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:15:19.792925 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:19.792893 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:15:19.793169 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:19.793144 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:15:19.793884 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:19.793864 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:15:20.415164 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:20.415125 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" event={"ID":"fcf64787-d325-4b03-a982-3dcb538b0d5e","Type":"ContainerDied","Data":"7cb5a3fed61ababe54fe82c2f03be8c3c5935c1fc325beec8bdb69aeb62f4b56"} Jul 07 00:15:20.415164 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:20.415146 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s" Jul 07 00:15:20.415428 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:20.415178 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cb5a3fed61ababe54fe82c2f03be8c3c5935c1fc325beec8bdb69aeb62f4b56" Jul 07 00:15:20.441322 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:20.441247 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s"] Jul 07 00:15:20.446552 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:20.446530 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-mt-c75b7974-29723055-kqk6s"] Jul 07 00:15:22.181340 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:22.181305 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" path="/var/lib/kubelet/pods/fcf64787-d325-4b03-a982-3dcb538b0d5e/volumes" Jul 07 00:15:24.800340 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:24.800309 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:15:35.390968 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:35.390937 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:15:44.703381 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:15:44.703346 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:16:21.923298 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:21.923266 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:16:24.953701 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:24.953661 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq"] Jul 07 00:16:24.954172 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:24.953968 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" containerID="cri-o://c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b" gracePeriod=30 Jul 07 00:16:30.224935 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.224875 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:16:30.368530 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368469 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-socket\") pod \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " Jul 07 00:16:30.368727 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368554 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-envoy\") pod \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " Jul 07 00:16:30.368727 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368573 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istiod-ca-cert\") pod \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " Jul 07 00:16:30.368727 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368591 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-token\") pod \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " Jul 07 00:16:30.368727 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368607 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fptgf\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-kube-api-access-fptgf\") pod \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " Jul 07 00:16:30.368727 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368638 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-podinfo\") pod \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " Jul 07 00:16:30.368727 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368681 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-certs\") pod \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " Jul 07 00:16:30.369058 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368754 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-data\") pod \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " Jul 07 00:16:30.369058 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368755 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8dd1fca0-a44a-48e4-9363-4f1dccc89b24" (UID: "8dd1fca0-a44a-48e4-9363-4f1dccc89b24"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:16:30.369058 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.368788 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-credential-socket\") pod \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\" (UID: \"8dd1fca0-a44a-48e4-9363-4f1dccc89b24\") " Jul 07 00:16:30.369058 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.369005 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8dd1fca0-a44a-48e4-9363-4f1dccc89b24" (UID: "8dd1fca0-a44a-48e4-9363-4f1dccc89b24"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:16:30.369058 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.369038 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8dd1fca0-a44a-48e4-9363-4f1dccc89b24" (UID: "8dd1fca0-a44a-48e4-9363-4f1dccc89b24"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:16:30.369330 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.369213 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8dd1fca0-a44a-48e4-9363-4f1dccc89b24" (UID: "8dd1fca0-a44a-48e4-9363-4f1dccc89b24"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:16:30.369330 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.369262 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:16:30.369330 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.369283 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:16:30.369330 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.369297 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:16:30.369814 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.369794 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-data" (OuterVolumeSpecName: "istio-data") pod "8dd1fca0-a44a-48e4-9363-4f1dccc89b24" (UID: "8dd1fca0-a44a-48e4-9363-4f1dccc89b24"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:16:30.370889 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.370865 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-token" (OuterVolumeSpecName: "istio-token") pod "8dd1fca0-a44a-48e4-9363-4f1dccc89b24" (UID: "8dd1fca0-a44a-48e4-9363-4f1dccc89b24"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:16:30.370972 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.370927 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8dd1fca0-a44a-48e4-9363-4f1dccc89b24" (UID: "8dd1fca0-a44a-48e4-9363-4f1dccc89b24"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:16:30.371255 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.371223 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8dd1fca0-a44a-48e4-9363-4f1dccc89b24" (UID: "8dd1fca0-a44a-48e4-9363-4f1dccc89b24"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:16:30.371355 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.371228 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-kube-api-access-fptgf" (OuterVolumeSpecName: "kube-api-access-fptgf") pod "8dd1fca0-a44a-48e4-9363-4f1dccc89b24" (UID: "8dd1fca0-a44a-48e4-9363-4f1dccc89b24"). InnerVolumeSpecName "kube-api-access-fptgf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:16:30.469898 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.469866 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:16:30.469898 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.469892 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:16:30.469898 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.469901 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:16:30.470114 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.469909 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:16:30.470114 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.469917 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fptgf\" (UniqueName: \"kubernetes.io/projected/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-kube-api-access-fptgf\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:16:30.470114 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.469925 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8dd1fca0-a44a-48e4-9363-4f1dccc89b24-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:16:30.822657 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.822621 2583 generic.go:356] "Generic (PLEG): container finished" podID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerID="c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b" exitCode=0 Jul 07 00:16:30.822865 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.822688 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" Jul 07 00:16:30.822865 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.822711 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" event={"ID":"8dd1fca0-a44a-48e4-9363-4f1dccc89b24","Type":"ContainerDied","Data":"c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b"} Jul 07 00:16:30.822865 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.822811 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq" event={"ID":"8dd1fca0-a44a-48e4-9363-4f1dccc89b24","Type":"ContainerDied","Data":"53744da3e50fb1e739324df6a7ba9c2123d25e8c688bbcb30fbbf984ce309293"} Jul 07 00:16:30.822865 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.822843 2583 scope.go:117] "RemoveContainer" containerID="c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b" Jul 07 00:16:30.832822 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.832803 2583 scope.go:117] "RemoveContainer" containerID="7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2" Jul 07 00:16:30.841084 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.841066 2583 scope.go:117] "RemoveContainer" containerID="c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b" Jul 07 00:16:30.841354 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:16:30.841335 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b\": container with ID starting with c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b not found: ID does not exist" containerID="c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b" Jul 07 00:16:30.841405 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.841363 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b"} err="failed to get container status \"c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b\": rpc error: code = NotFound desc = could not find container \"c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b\": container with ID starting with c1cc0dfd9251d8d047523f3605a54990a0a7cf0d0ebaf185589f785ebee3091b not found: ID does not exist" Jul 07 00:16:30.841405 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.841381 2583 scope.go:117] "RemoveContainer" containerID="7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2" Jul 07 00:16:30.841628 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:16:30.841613 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2\": container with ID starting with 7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2 not found: ID does not exist" containerID="7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2" Jul 07 00:16:30.841669 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.841631 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2"} err="failed to get container status \"7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2\": rpc error: code = NotFound desc = could not find container \"7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2\": container with ID starting with 7d372525d483d562b8d50838f6c9706570d975c1bfd96683407d7192ac3587b2 not found: ID does not exist" Jul 07 00:16:30.847751 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.847720 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq"] Jul 07 00:16:30.850171 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:30.850150 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-c75b7974-openshift-default-7f459bd695-ws7kq"] Jul 07 00:16:32.182449 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:32.182371 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" path="/var/lib/kubelet/pods/8dd1fca0-a44a-48e4-9363-4f1dccc89b24/volumes" Jul 07 00:16:34.402650 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:34.402616 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:16:43.306672 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:16:43.306629 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:17:19.609535 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.609490 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr"] Jul 07 00:17:19.610016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.609917 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerName="cleanup" Jul 07 00:17:19.610016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.609931 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerName="cleanup" Jul 07 00:17:19.610016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.609945 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" Jul 07 00:17:19.610016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.609951 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" Jul 07 00:17:19.610016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.609966 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerName="cleanup" Jul 07 00:17:19.610016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.609972 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerName="cleanup" Jul 07 00:17:19.610016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.609990 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54cc54a8-c744-462a-99d3-a9b43a4de909" containerName="cleanup" Jul 07 00:17:19.610016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.609996 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="54cc54a8-c744-462a-99d3-a9b43a4de909" containerName="cleanup" Jul 07 00:17:19.610403 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.610061 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerName="cleanup" Jul 07 00:17:19.610403 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.610070 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" Jul 07 00:17:19.610403 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.610075 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" Jul 07 00:17:19.610403 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.610084 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerName="cleanup" Jul 07 00:17:19.610403 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.610092 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="54cc54a8-c744-462a-99d3-a9b43a4de909" containerName="cleanup" Jul 07 00:17:19.613522 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.613500 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.616029 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.616006 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-30509d-openshift-default-dockercfg-fpz6x\"" Jul 07 00:17:19.627511 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.627446 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr"] Jul 07 00:17:19.631292 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.631272 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-istio-token\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.631387 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.631306 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-certs\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.631387 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.631373 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6127170-78c4-4021-82df-983d7d243ef7-istiod-ca-cert\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.631464 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.631415 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-socket\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.631464 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.631454 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpwtk\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-kube-api-access-kpwtk\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.631552 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.631485 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-credential-socket\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.631552 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.631509 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-envoy\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.631552 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.631528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6127170-78c4-4021-82df-983d7d243ef7-istio-podinfo\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.631552 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.631546 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-data\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.732465 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.732432 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-istio-token\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.732651 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.732496 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-certs\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.732651 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.732545 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6127170-78c4-4021-82df-983d7d243ef7-istiod-ca-cert\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.732651 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.732598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-socket\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.732777 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.732726 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpwtk\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-kube-api-access-kpwtk\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.732819 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.732787 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-credential-socket\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.732866 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.732823 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-envoy\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.732993 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.732969 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-certs\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.733132 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.733019 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6127170-78c4-4021-82df-983d7d243ef7-istio-podinfo\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.733132 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.732988 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-socket\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.733132 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.733066 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-data\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.733309 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.733266 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-credential-socket\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.733377 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.733328 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-data\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.733437 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.733417 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6127170-78c4-4021-82df-983d7d243ef7-istiod-ca-cert\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.735128 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.735095 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-envoy\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.735345 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.735326 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6127170-78c4-4021-82df-983d7d243ef7-istio-podinfo\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.741161 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.741132 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-istio-token\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.741467 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.741446 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpwtk\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-kube-api-access-kpwtk\") pod \"e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:19.923018 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:19.922938 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:20.066300 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:20.066276 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr"] Jul 07 00:17:20.067432 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:17:20.067401 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6127170_78c4_4021_82df_983d7d243ef7.slice/crio-6bae101591efe523b3dd61e98bb3b6ce3261745c30168c472324edd09bc36e64 WatchSource:0}: Error finding container 6bae101591efe523b3dd61e98bb3b6ce3261745c30168c472324edd09bc36e64: Status 404 returned error can't find the container with id 6bae101591efe523b3dd61e98bb3b6ce3261745c30168c472324edd09bc36e64 Jul 07 00:17:20.069736 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:20.069702 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:20.069825 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:20.069766 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:20.069825 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:20.069794 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:20.106042 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:20.106005 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" event={"ID":"a6127170-78c4-4021-82df-983d7d243ef7","Type":"ContainerStarted","Data":"6bae101591efe523b3dd61e98bb3b6ce3261745c30168c472324edd09bc36e64"} Jul 07 00:17:21.113374 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:21.113336 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" event={"ID":"a6127170-78c4-4021-82df-983d7d243ef7","Type":"ContainerStarted","Data":"43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4"} Jul 07 00:17:21.134360 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:21.134301 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" podStartSLOduration=2.13428495 podStartE2EDuration="2.13428495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:17:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:17:21.131940205 +0000 UTC m=+1751.394612401" watchObservedRunningTime="2026-07-07 00:17:21.13428495 +0000 UTC m=+1751.396957125" Jul 07 00:17:21.924047 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:21.924002 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:21.928961 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:21.928937 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:22.119343 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:22.119314 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:22.120447 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:22.120420 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:28.126349 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.126316 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8"] Jul 07 00:17:28.126760 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.126746 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerName="cleanup" Jul 07 00:17:28.126822 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.126762 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerName="cleanup" Jul 07 00:17:28.126822 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.126772 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" Jul 07 00:17:28.126822 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.126777 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dd1fca0-a44a-48e4-9363-4f1dccc89b24" containerName="istio-proxy" Jul 07 00:17:28.126914 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.126858 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcf64787-d325-4b03-a982-3dcb538b0d5e" containerName="cleanup" Jul 07 00:17:28.130122 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.130102 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.133288 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.133245 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-408dd6-openshift-default-dockercfg-kls79\"" Jul 07 00:17:28.135574 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.135549 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8"] Jul 07 00:17:28.217082 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.217026 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-data\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.217263 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.217094 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0208049e-d6ba-4093-a422-53b29e1833a0-istio-podinfo\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.217263 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.217146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h46r9\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-kube-api-access-h46r9\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.217263 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.217249 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-istio-token\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.217403 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.217308 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-socket\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.217403 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.217332 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-envoy\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.217403 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.217396 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-credential-socket\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.217570 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.217456 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0208049e-d6ba-4093-a422-53b29e1833a0-istiod-ca-cert\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.217570 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.217551 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-certs\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318390 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318358 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-istio-token\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318585 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318407 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-socket\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318585 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318425 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-envoy\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318585 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318458 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-credential-socket\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318585 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318509 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0208049e-d6ba-4093-a422-53b29e1833a0-istiod-ca-cert\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318585 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318542 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-certs\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318585 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318567 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-data\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318932 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318638 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0208049e-d6ba-4093-a422-53b29e1833a0-istio-podinfo\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318932 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318671 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h46r9\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-kube-api-access-h46r9\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318932 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318853 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-socket\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.318932 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.318879 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-credential-socket\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.319108 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.319013 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-certs\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.319108 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.319080 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-data\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.319310 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.319293 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0208049e-d6ba-4093-a422-53b29e1833a0-istiod-ca-cert\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.320755 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.320733 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-envoy\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.321184 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.321154 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0208049e-d6ba-4093-a422-53b29e1833a0-istio-podinfo\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.328698 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.328675 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h46r9\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-kube-api-access-h46r9\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.328817 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.328764 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-istio-token\") pod \"e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.442619 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.442530 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:28.588737 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.588704 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8"] Jul 07 00:17:28.589449 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:17:28.589426 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0208049e_d6ba_4093_a422_53b29e1833a0.slice/crio-5589e5f026591ca6916a618b911b03de7171a2fe33b6cd27376d54b3476e9023 WatchSource:0}: Error finding container 5589e5f026591ca6916a618b911b03de7171a2fe33b6cd27376d54b3476e9023: Status 404 returned error can't find the container with id 5589e5f026591ca6916a618b911b03de7171a2fe33b6cd27376d54b3476e9023 Jul 07 00:17:28.591746 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.591709 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:28.591849 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.591778 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:28.591849 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:28.591807 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:29.162170 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:29.162138 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" event={"ID":"0208049e-d6ba-4093-a422-53b29e1833a0","Type":"ContainerStarted","Data":"2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79"} Jul 07 00:17:29.162608 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:29.162186 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" event={"ID":"0208049e-d6ba-4093-a422-53b29e1833a0","Type":"ContainerStarted","Data":"5589e5f026591ca6916a618b911b03de7171a2fe33b6cd27376d54b3476e9023"} Jul 07 00:17:29.180014 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:29.179960 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" podStartSLOduration=1.179938695 podStartE2EDuration="1.179938695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:17:29.179586596 +0000 UTC m=+1759.442258804" watchObservedRunningTime="2026-07-07 00:17:29.179938695 +0000 UTC m=+1759.442610892" Jul 07 00:17:29.443034 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:29.442951 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:29.448142 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:29.448123 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:30.167973 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:30.167937 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:30.168894 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:30.168871 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:30.207871 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:30.207842 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:17:37.985232 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:37.984804 2583 patch_prober.go:28] interesting pod/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:56628->10.133.0.62:15021: read: connection reset by peer" start-of-body= Jul 07 00:17:37.985232 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:37.984887 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": read tcp 10.133.0.2:56628->10.133.0.62:15021: read: connection reset by peer" Jul 07 00:17:38.217948 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:38.217868 2583 generic.go:356] "Generic (PLEG): container finished" podID="a6127170-78c4-4021-82df-983d7d243ef7" containerID="43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4" exitCode=0 Jul 07 00:17:38.218097 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:38.217947 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" event={"ID":"a6127170-78c4-4021-82df-983d7d243ef7","Type":"ContainerDied","Data":"43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4"} Jul 07 00:17:38.218396 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:38.218383 2583 scope.go:117] "RemoveContainer" containerID="43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4" Jul 07 00:17:38.219443 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:38.219416 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:38.219534 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:38.219507 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:38.219581 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:38.219552 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:39.247022 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:39.239150 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" event={"ID":"a6127170-78c4-4021-82df-983d7d243ef7","Type":"ContainerStarted","Data":"59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58"} Jul 07 00:17:39.923486 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:39.923436 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:39.923787 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:39.923756 2583 patch_prober.go:28] interesting pod/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 00:17:39.924006 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:39.923975 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 00:17:40.613252 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:40.613214 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:17:40.923901 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:40.923811 2583 patch_prober.go:28] interesting pod/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 00:17:40.923901 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:40.923870 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 00:17:41.924359 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:41.924324 2583 patch_prober.go:28] interesting pod/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 00:17:41.924803 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:41.924377 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 00:17:42.923657 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:42.923615 2583 patch_prober.go:28] interesting pod/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 00:17:42.923835 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:42.923684 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 00:17:43.923947 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:43.923907 2583 patch_prober.go:28] interesting pod/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body= Jul 07 00:17:43.924347 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:43.923968 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" Jul 07 00:17:44.926948 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:44.926870 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:44.927458 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:44.927256 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:44.927878 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:44.927862 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:17:45.169491 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:45.169449 2583 patch_prober.go:28] interesting pod/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 07 00:17:45.169707 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:45.169552 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 07 00:17:45.294592 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:45.294555 2583 generic.go:356] "Generic (PLEG): container finished" podID="0208049e-d6ba-4093-a422-53b29e1833a0" containerID="2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79" exitCode=0 Jul 07 00:17:45.294765 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:45.294637 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" event={"ID":"0208049e-d6ba-4093-a422-53b29e1833a0","Type":"ContainerDied","Data":"2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79"} Jul 07 00:17:45.295193 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:45.295179 2583 scope.go:117] "RemoveContainer" containerID="2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79" Jul 07 00:17:45.296115 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:45.296090 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:45.296165 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:45.296151 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:45.296201 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:45.296182 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:17:46.303132 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:46.303100 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" event={"ID":"0208049e-d6ba-4093-a422-53b29e1833a0","Type":"ContainerStarted","Data":"4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387"} Jul 07 00:17:46.443662 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:46.443618 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:46.444103 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:46.444073 2583 patch_prober.go:28] interesting pod/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 07 00:17:46.444230 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:46.444147 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 07 00:17:47.443232 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:47.443189 2583 patch_prober.go:28] interesting pod/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 07 00:17:47.443904 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:47.443867 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 07 00:17:48.443986 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:48.443510 2583 patch_prober.go:28] interesting pod/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 07 00:17:48.443986 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:48.443583 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 07 00:17:49.443589 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:49.443541 2583 patch_prober.go:28] interesting pod/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 07 00:17:49.443823 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:49.443619 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 07 00:17:50.447632 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:50.447587 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:50.448040 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:50.447850 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:17:50.448528 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:17:50.448511 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:18:05.603289 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:18:05.603207 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:18:16.802129 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:18:16.802090 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:18:44.202084 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:18:44.202046 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:18:56.505618 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:18:56.505576 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:19:11.200187 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:11.200142 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:19:43.258585 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:43.258493 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr"] Jul 07 00:19:43.259176 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:43.258747 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" containerID="cri-o://59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58" gracePeriod=30 Jul 07 00:19:44.929155 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:44.929119 2583 patch_prober.go:28] interesting pod/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 00:19:44.929657 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:44.929222 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 00:19:46.198008 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:46.197972 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:19:48.528860 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.528802 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:19:48.671753 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.671720 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-envoy\") pod \"a6127170-78c4-4021-82df-983d7d243ef7\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " Jul 07 00:19:48.671956 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.671772 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-certs\") pod \"a6127170-78c4-4021-82df-983d7d243ef7\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " Jul 07 00:19:48.671956 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.671793 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-istio-token\") pod \"a6127170-78c4-4021-82df-983d7d243ef7\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " Jul 07 00:19:48.671956 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.671807 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-socket\") pod \"a6127170-78c4-4021-82df-983d7d243ef7\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " Jul 07 00:19:48.671956 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.671829 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-credential-socket\") pod \"a6127170-78c4-4021-82df-983d7d243ef7\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " Jul 07 00:19:48.671956 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.671867 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6127170-78c4-4021-82df-983d7d243ef7-istiod-ca-cert\") pod \"a6127170-78c4-4021-82df-983d7d243ef7\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " Jul 07 00:19:48.671956 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.671896 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpwtk\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-kube-api-access-kpwtk\") pod \"a6127170-78c4-4021-82df-983d7d243ef7\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " Jul 07 00:19:48.672277 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672055 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6127170-78c4-4021-82df-983d7d243ef7-istio-podinfo\") pod \"a6127170-78c4-4021-82df-983d7d243ef7\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " Jul 07 00:19:48.672277 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672063 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a6127170-78c4-4021-82df-983d7d243ef7" (UID: "a6127170-78c4-4021-82df-983d7d243ef7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:48.672277 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672134 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-data\") pod \"a6127170-78c4-4021-82df-983d7d243ef7\" (UID: \"a6127170-78c4-4021-82df-983d7d243ef7\") " Jul 07 00:19:48.672277 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672154 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a6127170-78c4-4021-82df-983d7d243ef7" (UID: "a6127170-78c4-4021-82df-983d7d243ef7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:48.672277 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672219 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a6127170-78c4-4021-82df-983d7d243ef7" (UID: "a6127170-78c4-4021-82df-983d7d243ef7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:48.672556 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672342 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6127170-78c4-4021-82df-983d7d243ef7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a6127170-78c4-4021-82df-983d7d243ef7" (UID: "a6127170-78c4-4021-82df-983d7d243ef7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:19:48.672653 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672633 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:48.672707 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672659 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:48.672707 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672673 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:48.672707 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.672688 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6127170-78c4-4021-82df-983d7d243ef7-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:48.673335 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.673307 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-data" (OuterVolumeSpecName: "istio-data") pod "a6127170-78c4-4021-82df-983d7d243ef7" (UID: "a6127170-78c4-4021-82df-983d7d243ef7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:48.674087 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.674055 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-istio-token" (OuterVolumeSpecName: "istio-token") pod "a6127170-78c4-4021-82df-983d7d243ef7" (UID: "a6127170-78c4-4021-82df-983d7d243ef7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:19:48.674181 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.674118 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a6127170-78c4-4021-82df-983d7d243ef7" (UID: "a6127170-78c4-4021-82df-983d7d243ef7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:48.674181 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.674138 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6127170-78c4-4021-82df-983d7d243ef7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a6127170-78c4-4021-82df-983d7d243ef7" (UID: "a6127170-78c4-4021-82df-983d7d243ef7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:19:48.674256 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.674238 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-kube-api-access-kpwtk" (OuterVolumeSpecName: "kube-api-access-kpwtk") pod "a6127170-78c4-4021-82df-983d7d243ef7" (UID: "a6127170-78c4-4021-82df-983d7d243ef7"). InnerVolumeSpecName "kube-api-access-kpwtk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:19:48.773924 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.773887 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6127170-78c4-4021-82df-983d7d243ef7-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:48.773924 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.773916 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:48.773924 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.773924 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6127170-78c4-4021-82df-983d7d243ef7-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:48.773924 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.773932 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:48.774184 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:48.773940 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpwtk\" (UniqueName: \"kubernetes.io/projected/a6127170-78c4-4021-82df-983d7d243ef7-kube-api-access-kpwtk\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:49.054378 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.054335 2583 generic.go:356] "Generic (PLEG): container finished" podID="a6127170-78c4-4021-82df-983d7d243ef7" containerID="59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58" exitCode=0 Jul 07 00:19:49.054574 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.054402 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" Jul 07 00:19:49.054574 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.054426 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" event={"ID":"a6127170-78c4-4021-82df-983d7d243ef7","Type":"ContainerDied","Data":"59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58"} Jul 07 00:19:49.054574 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.054508 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr" event={"ID":"a6127170-78c4-4021-82df-983d7d243ef7","Type":"ContainerDied","Data":"6bae101591efe523b3dd61e98bb3b6ce3261745c30168c472324edd09bc36e64"} Jul 07 00:19:49.054574 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.054536 2583 scope.go:117] "RemoveContainer" containerID="59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58" Jul 07 00:19:49.065151 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.065128 2583 scope.go:117] "RemoveContainer" containerID="43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4" Jul 07 00:19:49.073966 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.073948 2583 scope.go:117] "RemoveContainer" containerID="59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58" Jul 07 00:19:49.074267 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:19:49.074241 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58\": container with ID starting with 59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58 not found: ID does not exist" containerID="59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58" Jul 07 00:19:49.074311 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.074280 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58"} err="failed to get container status \"59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58\": rpc error: code = NotFound desc = could not find container \"59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58\": container with ID starting with 59c85b19099aa589d051f3b006b6ba665bd5b51b8a0726d28e0ab141e6c82f58 not found: ID does not exist" Jul 07 00:19:49.074311 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.074304 2583 scope.go:117] "RemoveContainer" containerID="43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4" Jul 07 00:19:49.074547 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:19:49.074528 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4\": container with ID starting with 43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4 not found: ID does not exist" containerID="43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4" Jul 07 00:19:49.074604 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.074552 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4"} err="failed to get container status \"43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4\": rpc error: code = NotFound desc = could not find container \"43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4\": container with ID starting with 43e01b4e00968b454d0aa0ce9b296992d11adaf82de80fe38d763acae1c24dd4 not found: ID does not exist" Jul 07 00:19:49.075871 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.075852 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr"] Jul 07 00:19:49.078830 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:49.078812 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-30509d-openshift-default-75f7c9b955-tlvnr"] Jul 07 00:19:50.182782 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:50.182747 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6127170-78c4-4021-82df-983d7d243ef7" path="/var/lib/kubelet/pods/a6127170-78c4-4021-82df-983d7d243ef7/volumes" Jul 07 00:19:50.402461 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:50.402432 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8"] Jul 07 00:19:50.402679 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:50.402657 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" containerID="cri-o://4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387" gracePeriod=30 Jul 07 00:19:50.450698 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:50.450624 2583 patch_prober.go:28] interesting pod/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 00:19:50.450698 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:50.450674 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 00:19:51.343253 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.343216 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf"] Jul 07 00:19:51.343774 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.343748 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" Jul 07 00:19:51.343774 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.343767 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" Jul 07 00:19:51.343919 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.343904 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" Jul 07 00:19:51.343983 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.343923 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" Jul 07 00:19:51.348750 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.348732 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.350722 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.350703 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4d772e-openshift-default-dockercfg-lmdk5\"" Jul 07 00:19:51.358826 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.358803 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf"] Jul 07 00:19:51.501043 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.501009 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-socket\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.501043 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.501041 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-credential-socket\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.501254 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.501062 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-certs\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.501254 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.501083 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-data\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.501254 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.501200 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wkvc\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-kube-api-access-7wkvc\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.501254 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.501233 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-token\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.501401 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.501257 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6367ff1-fae2-4641-b8b9-d540d4615de1-istiod-ca-cert\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.501401 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.501280 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-envoy\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.501401 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.501350 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-podinfo\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.602940 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.602856 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wkvc\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-kube-api-access-7wkvc\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.602940 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.602895 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-token\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.602940 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.602921 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6367ff1-fae2-4641-b8b9-d540d4615de1-istiod-ca-cert\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.602940 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.602938 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-envoy\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603286 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.602961 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-podinfo\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603286 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.603011 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-socket\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603286 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.603028 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-credential-socket\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603286 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.603046 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-certs\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603286 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.603078 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-data\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603591 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.603409 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-socket\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603591 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.603508 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-credential-socket\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603591 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.603531 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-data\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603763 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.603607 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-certs\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.603815 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.603760 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6367ff1-fae2-4641-b8b9-d540d4615de1-istiod-ca-cert\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.605302 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.605278 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-envoy\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.605404 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.605395 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-podinfo\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.610845 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.610823 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-token\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.610942 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.610903 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wkvc\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-kube-api-access-7wkvc\") pod \"e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.660043 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.660016 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:51.800580 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.800555 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf"] Jul 07 00:19:51.801736 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:19:51.801709 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6367ff1_fae2_4641_b8b9_d540d4615de1.slice/crio-c264039fdc536d6bbd6c086a565d2afc2a91b63079a364b8728840f382b4298e WatchSource:0}: Error finding container c264039fdc536d6bbd6c086a565d2afc2a91b63079a364b8728840f382b4298e: Status 404 returned error can't find the container with id c264039fdc536d6bbd6c086a565d2afc2a91b63079a364b8728840f382b4298e Jul 07 00:19:51.804161 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.804116 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:19:51.804254 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.804186 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:19:51.804254 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:51.804217 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:19:52.075713 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:52.075675 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" event={"ID":"a6367ff1-fae2-4641-b8b9-d540d4615de1","Type":"ContainerStarted","Data":"e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d"} Jul 07 00:19:52.075916 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:52.075737 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" event={"ID":"a6367ff1-fae2-4641-b8b9-d540d4615de1","Type":"ContainerStarted","Data":"c264039fdc536d6bbd6c086a565d2afc2a91b63079a364b8728840f382b4298e"} Jul 07 00:19:52.097944 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:52.097892 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" podStartSLOduration=1.09787841 podStartE2EDuration="1.09787841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:19:52.09522926 +0000 UTC m=+1902.357901454" watchObservedRunningTime="2026-07-07 00:19:52.09787841 +0000 UTC m=+1902.360550604" Jul 07 00:19:52.660802 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:52.660764 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:52.661876 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:52.661853 2583 patch_prober.go:28] interesting pod/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 07 00:19:52.661964 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:52.661902 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 07 00:19:53.664712 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:53.664681 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:54.088260 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:54.088224 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:54.089319 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:54.089299 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:19:55.662326 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.662261 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:19:55.747407 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747371 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0208049e-d6ba-4093-a422-53b29e1833a0-istio-podinfo\") pod \"0208049e-d6ba-4093-a422-53b29e1833a0\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " Jul 07 00:19:55.747643 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747423 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-socket\") pod \"0208049e-d6ba-4093-a422-53b29e1833a0\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " Jul 07 00:19:55.747643 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747448 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-certs\") pod \"0208049e-d6ba-4093-a422-53b29e1833a0\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " Jul 07 00:19:55.747643 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747466 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0208049e-d6ba-4093-a422-53b29e1833a0-istiod-ca-cert\") pod \"0208049e-d6ba-4093-a422-53b29e1833a0\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " Jul 07 00:19:55.747643 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747573 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-istio-token\") pod \"0208049e-d6ba-4093-a422-53b29e1833a0\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " Jul 07 00:19:55.747643 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747611 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-envoy\") pod \"0208049e-d6ba-4093-a422-53b29e1833a0\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " Jul 07 00:19:55.747915 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747655 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h46r9\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-kube-api-access-h46r9\") pod \"0208049e-d6ba-4093-a422-53b29e1833a0\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " Jul 07 00:19:55.747915 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747703 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-credential-socket\") pod \"0208049e-d6ba-4093-a422-53b29e1833a0\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " Jul 07 00:19:55.747915 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747774 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-data\") pod \"0208049e-d6ba-4093-a422-53b29e1833a0\" (UID: \"0208049e-d6ba-4093-a422-53b29e1833a0\") " Jul 07 00:19:55.747915 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747774 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0208049e-d6ba-4093-a422-53b29e1833a0" (UID: "0208049e-d6ba-4093-a422-53b29e1833a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:55.747915 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.747910 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0208049e-d6ba-4093-a422-53b29e1833a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0208049e-d6ba-4093-a422-53b29e1833a0" (UID: "0208049e-d6ba-4093-a422-53b29e1833a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:19:55.748185 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.748054 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0208049e-d6ba-4093-a422-53b29e1833a0" (UID: "0208049e-d6ba-4093-a422-53b29e1833a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:55.748185 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.748145 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0208049e-d6ba-4093-a422-53b29e1833a0" (UID: "0208049e-d6ba-4093-a422-53b29e1833a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:55.748299 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.748255 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:55.748299 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.748272 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:55.748299 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.748286 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0208049e-d6ba-4093-a422-53b29e1833a0-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:55.748460 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.748299 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:55.748637 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.748615 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "0208049e-d6ba-4093-a422-53b29e1833a0" (UID: "0208049e-d6ba-4093-a422-53b29e1833a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:55.749839 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.749805 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0208049e-d6ba-4093-a422-53b29e1833a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0208049e-d6ba-4093-a422-53b29e1833a0" (UID: "0208049e-d6ba-4093-a422-53b29e1833a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:19:55.749839 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.749813 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "0208049e-d6ba-4093-a422-53b29e1833a0" (UID: "0208049e-d6ba-4093-a422-53b29e1833a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:19:55.750187 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.750166 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-kube-api-access-h46r9" (OuterVolumeSpecName: "kube-api-access-h46r9") pod "0208049e-d6ba-4093-a422-53b29e1833a0" (UID: "0208049e-d6ba-4093-a422-53b29e1833a0"). InnerVolumeSpecName "kube-api-access-h46r9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:19:55.750248 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.750177 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0208049e-d6ba-4093-a422-53b29e1833a0" (UID: "0208049e-d6ba-4093-a422-53b29e1833a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:19:55.849035 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.848993 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0208049e-d6ba-4093-a422-53b29e1833a0-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:55.849035 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.849028 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:55.849035 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.849038 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:55.849035 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.849046 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h46r9\" (UniqueName: \"kubernetes.io/projected/0208049e-d6ba-4093-a422-53b29e1833a0-kube-api-access-h46r9\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:55.849306 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:55.849056 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0208049e-d6ba-4093-a422-53b29e1833a0-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:19:56.102217 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.102181 2583 generic.go:356] "Generic (PLEG): container finished" podID="0208049e-d6ba-4093-a422-53b29e1833a0" containerID="4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387" exitCode=0 Jul 07 00:19:56.102422 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.102237 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" Jul 07 00:19:56.102422 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.102268 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" event={"ID":"0208049e-d6ba-4093-a422-53b29e1833a0","Type":"ContainerDied","Data":"4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387"} Jul 07 00:19:56.102422 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.102320 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8" event={"ID":"0208049e-d6ba-4093-a422-53b29e1833a0","Type":"ContainerDied","Data":"5589e5f026591ca6916a618b911b03de7171a2fe33b6cd27376d54b3476e9023"} Jul 07 00:19:56.102422 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.102349 2583 scope.go:117] "RemoveContainer" containerID="4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387" Jul 07 00:19:56.112723 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.112705 2583 scope.go:117] "RemoveContainer" containerID="2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79" Jul 07 00:19:56.121066 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.121047 2583 scope.go:117] "RemoveContainer" containerID="4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387" Jul 07 00:19:56.121354 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:19:56.121326 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387\": container with ID starting with 4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387 not found: ID does not exist" containerID="4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387" Jul 07 00:19:56.121464 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.121365 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387"} err="failed to get container status \"4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387\": rpc error: code = NotFound desc = could not find container \"4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387\": container with ID starting with 4466d5ab21f42bee06eb8684435055bb46d6688bc03c01d2988632c5671da387 not found: ID does not exist" Jul 07 00:19:56.121464 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.121387 2583 scope.go:117] "RemoveContainer" containerID="2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79" Jul 07 00:19:56.121876 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:19:56.121853 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79\": container with ID starting with 2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79 not found: ID does not exist" containerID="2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79" Jul 07 00:19:56.121980 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.121885 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79"} err="failed to get container status \"2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79\": rpc error: code = NotFound desc = could not find container \"2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79\": container with ID starting with 2de410a9149d3e30301afed99372d2b12894172452c874cf9c5a0a2ef6fcca79 not found: ID does not exist" Jul 07 00:19:56.124653 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.124632 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8"] Jul 07 00:19:56.128264 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.128244 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-408dd6-openshift-default-67c5ffff4c-7s7x8"] Jul 07 00:19:56.182600 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:56.182565 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" path="/var/lib/kubelet/pods/0208049e-d6ba-4093-a422-53b29e1833a0/volumes" Jul 07 00:19:58.992200 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:58.992161 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:19:59.276956 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.276912 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj"] Jul 07 00:19:59.277367 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.277350 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" Jul 07 00:19:59.277367 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.277366 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" Jul 07 00:19:59.277559 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.277397 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" Jul 07 00:19:59.277559 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.277403 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6127170-78c4-4021-82df-983d7d243ef7" containerName="istio-proxy" Jul 07 00:19:59.277559 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.277507 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" Jul 07 00:19:59.277559 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.277517 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" Jul 07 00:19:59.282128 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.282108 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.284138 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.284117 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-5b1463-openshift-default-dockercfg-5pnmn\"" Jul 07 00:19:59.292733 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.292708 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj"] Jul 07 00:19:59.381866 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.381835 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-envoy\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.382049 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.381882 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istiod-ca-cert\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.382049 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.381905 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgn7x\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-kube-api-access-qgn7x\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.382049 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.381980 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-data\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.382049 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.382033 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-credential-socket\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.382234 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.382054 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-certs\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.382234 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.382082 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-token\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.382234 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.382101 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-socket\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.382234 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.382184 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-podinfo\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483278 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483240 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istiod-ca-cert\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483452 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483293 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgn7x\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-kube-api-access-qgn7x\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483452 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483358 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-data\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483452 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483409 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-credential-socket\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483452 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483436 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-certs\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483704 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483495 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-token\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483704 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483523 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-socket\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483704 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483551 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-podinfo\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483704 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-envoy\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.483915 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483870 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-certs\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.484008 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.483984 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istiod-ca-cert\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.484130 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.484101 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-credential-socket\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.484254 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.484212 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-socket\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.484396 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.484375 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-data\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.486442 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.486409 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-podinfo\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.486646 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.486619 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-envoy\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.495604 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.495573 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-token\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.495936 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.495918 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgn7x\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-kube-api-access-qgn7x\") pod \"e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.593110 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.593032 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:19:59.741575 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:19:59.741546 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc42e891c_5dbf_4260_84d6_bbb0fdbe2638.slice/crio-5c22a37abb03283c85b8fc9d12ec60335e1b4776555b8d6709aa868b4b048b15 WatchSource:0}: Error finding container 5c22a37abb03283c85b8fc9d12ec60335e1b4776555b8d6709aa868b4b048b15: Status 404 returned error can't find the container with id 5c22a37abb03283c85b8fc9d12ec60335e1b4776555b8d6709aa868b4b048b15 Jul 07 00:19:59.741984 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.741959 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj"] Jul 07 00:19:59.743686 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.743646 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:19:59.743765 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.743729 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:19:59.743805 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:19:59.743770 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:00.128071 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:00.128036 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" event={"ID":"c42e891c-5dbf-4260-84d6-bbb0fdbe2638","Type":"ContainerStarted","Data":"2d4446cfec8cca6c828d4e041b0e630c7abed95760309d213c52b3ee5d7682cf"} Jul 07 00:20:00.128583 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:00.128093 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" event={"ID":"c42e891c-5dbf-4260-84d6-bbb0fdbe2638","Type":"ContainerStarted","Data":"5c22a37abb03283c85b8fc9d12ec60335e1b4776555b8d6709aa868b4b048b15"} Jul 07 00:20:00.150498 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:00.147228 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" podStartSLOduration=1.147205609 podStartE2EDuration="1.147205609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:20:00.145617416 +0000 UTC m=+1910.408289611" watchObservedRunningTime="2026-07-07 00:20:00.147205609 +0000 UTC m=+1910.409877805" Jul 07 00:20:00.594089 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:00.594050 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:00.598690 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:00.598663 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:01.136221 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:01.136183 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:01.141040 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:01.141012 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:07.201953 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:07.201852 2583 generic.go:356] "Generic (PLEG): container finished" podID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerID="e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d" exitCode=0 Jul 07 00:20:07.201953 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:07.201914 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" event={"ID":"a6367ff1-fae2-4641-b8b9-d540d4615de1","Type":"ContainerDied","Data":"e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d"} Jul 07 00:20:07.202502 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:07.202287 2583 scope.go:117] "RemoveContainer" containerID="e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d" Jul 07 00:20:07.203316 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:07.203283 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:07.203384 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:07.203354 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:07.203420 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:07.203393 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:08.209543 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:08.209505 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" event={"ID":"a6367ff1-fae2-4641-b8b9-d540d4615de1","Type":"ContainerStarted","Data":"fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c"} Jul 07 00:20:08.660658 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:08.660615 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:20:08.661892 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:08.661865 2583 patch_prober.go:28] interesting pod/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 07 00:20:08.662028 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:08.661938 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 07 00:20:09.089153 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:09.089117 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:20:09.660601 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:09.660564 2583 patch_prober.go:28] interesting pod/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 07 00:20:09.661011 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:09.660617 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 07 00:20:10.046695 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:10.046638 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:20:10.661222 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:10.661177 2583 patch_prober.go:28] interesting pod/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 07 00:20:10.661732 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:10.661275 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 07 00:20:11.661202 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:11.661146 2583 patch_prober.go:28] interesting pod/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 07 00:20:11.661425 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:11.661236 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 07 00:20:12.664318 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:12.664281 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:20:12.665443 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:12.665423 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:20:17.177649 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:17.177614 2583 patch_prober.go:28] interesting pod/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 00:20:17.178086 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:17.177678 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 00:20:18.289433 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:18.289397 2583 generic.go:356] "Generic (PLEG): container finished" podID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerID="2d4446cfec8cca6c828d4e041b0e630c7abed95760309d213c52b3ee5d7682cf" exitCode=0 Jul 07 00:20:18.289987 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:18.289487 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" event={"ID":"c42e891c-5dbf-4260-84d6-bbb0fdbe2638","Type":"ContainerDied","Data":"2d4446cfec8cca6c828d4e041b0e630c7abed95760309d213c52b3ee5d7682cf"} Jul 07 00:20:18.289987 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:18.289972 2583 scope.go:117] "RemoveContainer" containerID="2d4446cfec8cca6c828d4e041b0e630c7abed95760309d213c52b3ee5d7682cf" Jul 07 00:20:18.291004 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:18.290976 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:18.291098 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:18.291055 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:18.291166 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:18.291107 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:19.297032 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:19.296998 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" event={"ID":"c42e891c-5dbf-4260-84d6-bbb0fdbe2638","Type":"ContainerStarted","Data":"3a55c83ebb49c2d72a6753b5e39068c59dbdd7f58c87dcc9e4a3585f10b82bda"} Jul 07 00:20:19.593397 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:19.593296 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:19.593602 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:19.593576 2583 patch_prober.go:28] interesting pod/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 07 00:20:19.593656 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:19.593631 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 07 00:20:20.594531 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:20.594457 2583 patch_prober.go:28] interesting pod/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 07 00:20:20.594949 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:20.594558 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 07 00:20:21.326849 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:21.326809 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf"] Jul 07 00:20:21.327055 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:21.327018 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" containerID="cri-o://fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c" gracePeriod=30 Jul 07 00:20:21.594150 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:21.594064 2583 patch_prober.go:28] interesting pod/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 07 00:20:21.594150 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:21.594128 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 07 00:20:21.602418 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:21.602385 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:20:22.593815 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:22.593775 2583 patch_prober.go:28] interesting pod/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 07 00:20:22.594039 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:22.593840 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 07 00:20:23.597423 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:23.597390 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:23.597855 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:23.597654 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:23.598326 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:23.598308 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:26.595657 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.595582 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:20:26.648542 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.648512 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-certs\") pod \"a6367ff1-fae2-4641-b8b9-d540d4615de1\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " Jul 07 00:20:26.648696 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.648550 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-token\") pod \"a6367ff1-fae2-4641-b8b9-d540d4615de1\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " Jul 07 00:20:26.648696 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.648637 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6367ff1-fae2-4641-b8b9-d540d4615de1-istiod-ca-cert\") pod \"a6367ff1-fae2-4641-b8b9-d540d4615de1\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " Jul 07 00:20:26.648834 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.648813 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-credential-socket\") pod \"a6367ff1-fae2-4641-b8b9-d540d4615de1\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " Jul 07 00:20:26.648834 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.648820 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a6367ff1-fae2-4641-b8b9-d540d4615de1" (UID: "a6367ff1-fae2-4641-b8b9-d540d4615de1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:26.648948 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.648869 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-data\") pod \"a6367ff1-fae2-4641-b8b9-d540d4615de1\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " Jul 07 00:20:26.648948 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.648915 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wkvc\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-kube-api-access-7wkvc\") pod \"a6367ff1-fae2-4641-b8b9-d540d4615de1\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " Jul 07 00:20:26.649060 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.648971 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-socket\") pod \"a6367ff1-fae2-4641-b8b9-d540d4615de1\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " Jul 07 00:20:26.649060 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.648975 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6367ff1-fae2-4641-b8b9-d540d4615de1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a6367ff1-fae2-4641-b8b9-d540d4615de1" (UID: "a6367ff1-fae2-4641-b8b9-d540d4615de1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:20:26.649060 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.649001 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-podinfo\") pod \"a6367ff1-fae2-4641-b8b9-d540d4615de1\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " Jul 07 00:20:26.649060 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.649030 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-envoy\") pod \"a6367ff1-fae2-4641-b8b9-d540d4615de1\" (UID: \"a6367ff1-fae2-4641-b8b9-d540d4615de1\") " Jul 07 00:20:26.649243 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.649221 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a6367ff1-fae2-4641-b8b9-d540d4615de1" (UID: "a6367ff1-fae2-4641-b8b9-d540d4615de1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:26.649385 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.649366 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:26.649459 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.649390 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:26.649459 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.649402 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6367ff1-fae2-4641-b8b9-d540d4615de1-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:26.649611 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.649595 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a6367ff1-fae2-4641-b8b9-d540d4615de1" (UID: "a6367ff1-fae2-4641-b8b9-d540d4615de1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:26.649757 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.649723 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-data" (OuterVolumeSpecName: "istio-data") pod "a6367ff1-fae2-4641-b8b9-d540d4615de1" (UID: "a6367ff1-fae2-4641-b8b9-d540d4615de1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:26.650941 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.650887 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-token" (OuterVolumeSpecName: "istio-token") pod "a6367ff1-fae2-4641-b8b9-d540d4615de1" (UID: "a6367ff1-fae2-4641-b8b9-d540d4615de1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:26.651195 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.651177 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a6367ff1-fae2-4641-b8b9-d540d4615de1" (UID: "a6367ff1-fae2-4641-b8b9-d540d4615de1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:26.651424 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.651397 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-kube-api-access-7wkvc" (OuterVolumeSpecName: "kube-api-access-7wkvc") pod "a6367ff1-fae2-4641-b8b9-d540d4615de1" (UID: "a6367ff1-fae2-4641-b8b9-d540d4615de1"). InnerVolumeSpecName "kube-api-access-7wkvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:26.651424 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.651401 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a6367ff1-fae2-4641-b8b9-d540d4615de1" (UID: "a6367ff1-fae2-4641-b8b9-d540d4615de1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:20:26.750066 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.750030 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:26.750066 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.750060 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:26.750066 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.750072 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wkvc\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-kube-api-access-7wkvc\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:26.750296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.750082 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:26.750296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.750090 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:26.750296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:26.750099 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6367ff1-fae2-4641-b8b9-d540d4615de1-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:27.349428 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.349396 2583 generic.go:356] "Generic (PLEG): container finished" podID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerID="fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c" exitCode=0 Jul 07 00:20:27.349729 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.349445 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" Jul 07 00:20:27.349729 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.349497 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" event={"ID":"a6367ff1-fae2-4641-b8b9-d540d4615de1","Type":"ContainerDied","Data":"fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c"} Jul 07 00:20:27.349729 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.349555 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf" event={"ID":"a6367ff1-fae2-4641-b8b9-d540d4615de1","Type":"ContainerDied","Data":"c264039fdc536d6bbd6c086a565d2afc2a91b63079a364b8728840f382b4298e"} Jul 07 00:20:27.349729 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.349580 2583 scope.go:117] "RemoveContainer" containerID="fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c" Jul 07 00:20:27.363136 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.363119 2583 scope.go:117] "RemoveContainer" containerID="e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d" Jul 07 00:20:27.371339 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.371322 2583 scope.go:117] "RemoveContainer" containerID="fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c" Jul 07 00:20:27.371627 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:20:27.371609 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c\": container with ID starting with fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c not found: ID does not exist" containerID="fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c" Jul 07 00:20:27.371682 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.371635 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c"} err="failed to get container status \"fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c\": rpc error: code = NotFound desc = could not find container \"fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c\": container with ID starting with fb8b9c127b38a260da8f637331614051380f93119c6729a3ed26437aa4e4b55c not found: ID does not exist" Jul 07 00:20:27.371682 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.371655 2583 scope.go:117] "RemoveContainer" containerID="e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d" Jul 07 00:20:27.371915 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:20:27.371896 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d\": container with ID starting with e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d not found: ID does not exist" containerID="e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d" Jul 07 00:20:27.371970 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.371923 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d"} err="failed to get container status \"e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d\": rpc error: code = NotFound desc = could not find container \"e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d\": container with ID starting with e912a928b4afb261f916bcba6ed67ccdb6f11e7d29ce96346817b53a629e271d not found: ID does not exist" Jul 07 00:20:27.375525 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.375503 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf"] Jul 07 00:20:27.377914 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:27.377895 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4d772e-openshift-default-65f5494d49-qlngf"] Jul 07 00:20:28.140343 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:28.140312 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj"] Jul 07 00:20:28.140752 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:28.140550 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" containerID="cri-o://3a55c83ebb49c2d72a6753b5e39068c59dbdd7f58c87dcc9e4a3585f10b82bda" gracePeriod=30 Jul 07 00:20:28.182044 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:28.182011 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" path="/var/lib/kubelet/pods/a6367ff1-fae2-4641-b8b9-d540d4615de1/volumes" Jul 07 00:20:29.066115 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.066088 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77"] Jul 07 00:20:29.066531 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.066518 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" Jul 07 00:20:29.066584 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.066532 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" Jul 07 00:20:29.066584 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.066549 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" Jul 07 00:20:29.066584 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.066554 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0208049e-d6ba-4093-a422-53b29e1833a0" containerName="istio-proxy" Jul 07 00:20:29.066679 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.066614 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" Jul 07 00:20:29.071276 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.071254 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.073178 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.073159 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-b60cd9-openshift-default-dockercfg-5glfx\"" Jul 07 00:20:29.082286 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.082266 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77"] Jul 07 00:20:29.173128 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.173092 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istiod-ca-cert\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.173610 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.173143 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-podinfo\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.173610 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.173296 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-certs\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.173610 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.173363 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64snj\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-kube-api-access-64snj\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.173610 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.173396 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-data\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.173610 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.173463 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-socket\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.173610 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.173565 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-credential-socket\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.173610 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.173598 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-envoy\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.173907 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.173632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-token\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.274547 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.274511 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-token\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.274742 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.274572 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istiod-ca-cert\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.274742 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.274597 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-podinfo\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.274742 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.274641 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-certs\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.274742 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.274688 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64snj\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-kube-api-access-64snj\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.274742 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.274715 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-data\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.275027 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.274759 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-socket\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.275027 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.274792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-credential-socket\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.275027 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.274819 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-envoy\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.275202 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.275101 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-certs\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.275202 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.275141 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-data\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.275202 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.275182 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-socket\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.275341 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.275262 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-credential-socket\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.275401 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.275381 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istiod-ca-cert\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.277059 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.277039 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-envoy\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.277159 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.277143 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-podinfo\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.281647 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.281621 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-token\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.282423 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.282406 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64snj\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-kube-api-access-64snj\") pod \"e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.382465 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.382383 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:29.535165 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.535137 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77"] Jul 07 00:20:29.537293 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:20:29.537264 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod780b0248_b6e4_4203_b2d1_e4f9be3c9dc7.slice/crio-55485e043bf0fb18b889772201f15bfa4d7c3cc822da86e138df993b25d1270e WatchSource:0}: Error finding container 55485e043bf0fb18b889772201f15bfa4d7c3cc822da86e138df993b25d1270e: Status 404 returned error can't find the container with id 55485e043bf0fb18b889772201f15bfa4d7c3cc822da86e138df993b25d1270e Jul 07 00:20:29.539866 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.539830 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:29.539983 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.539900 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:29.539983 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:29.539948 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:30.372000 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:30.371965 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" event={"ID":"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7","Type":"ContainerStarted","Data":"7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9"} Jul 07 00:20:30.372409 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:30.372019 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" event={"ID":"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7","Type":"ContainerStarted","Data":"55485e043bf0fb18b889772201f15bfa4d7c3cc822da86e138df993b25d1270e"} Jul 07 00:20:30.382753 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:30.382730 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:30.387735 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:30.387712 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:30.391181 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:30.391136 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" podStartSLOduration=1.391119771 podStartE2EDuration="1.391119771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:20:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:20:30.388669715 +0000 UTC m=+1940.651341909" watchObservedRunningTime="2026-07-07 00:20:30.391119771 +0000 UTC m=+1940.653791968" Jul 07 00:20:31.378150 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:31.378123 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:31.379030 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:31.379009 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:31.794085 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:31.794052 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:20:33.393361 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.393333 2583 generic.go:356] "Generic (PLEG): container finished" podID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerID="3a55c83ebb49c2d72a6753b5e39068c59dbdd7f58c87dcc9e4a3585f10b82bda" exitCode=0 Jul 07 00:20:33.393786 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.393405 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" event={"ID":"c42e891c-5dbf-4260-84d6-bbb0fdbe2638","Type":"ContainerDied","Data":"3a55c83ebb49c2d72a6753b5e39068c59dbdd7f58c87dcc9e4a3585f10b82bda"} Jul 07 00:20:33.393786 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.393503 2583 scope.go:117] "RemoveContainer" containerID="2d4446cfec8cca6c828d4e041b0e630c7abed95760309d213c52b3ee5d7682cf" Jul 07 00:20:33.393786 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.393501 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" event={"ID":"c42e891c-5dbf-4260-84d6-bbb0fdbe2638","Type":"ContainerDied","Data":"5c22a37abb03283c85b8fc9d12ec60335e1b4776555b8d6709aa868b4b048b15"} Jul 07 00:20:33.393786 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.393618 2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c22a37abb03283c85b8fc9d12ec60335e1b4776555b8d6709aa868b4b048b15" Jul 07 00:20:33.405281 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.405264 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:33.514553 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.514529 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-podinfo\") pod \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " Jul 07 00:20:33.514553 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.514562 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgn7x\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-kube-api-access-qgn7x\") pod \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " Jul 07 00:20:33.514769 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.514595 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-envoy\") pod \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " Jul 07 00:20:33.514769 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.514614 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-data\") pod \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " Jul 07 00:20:33.514769 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.514654 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istiod-ca-cert\") pod \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " Jul 07 00:20:33.514769 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.514709 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-socket\") pod \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " Jul 07 00:20:33.514769 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.514741 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-certs\") pod \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " Jul 07 00:20:33.514769 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.514765 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-token\") pod \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " Jul 07 00:20:33.515125 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.514809 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-credential-socket\") pod \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\" (UID: \"c42e891c-5dbf-4260-84d6-bbb0fdbe2638\") " Jul 07 00:20:33.515125 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.515026 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c42e891c-5dbf-4260-84d6-bbb0fdbe2638" (UID: "c42e891c-5dbf-4260-84d6-bbb0fdbe2638"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:33.515125 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.515045 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c42e891c-5dbf-4260-84d6-bbb0fdbe2638" (UID: "c42e891c-5dbf-4260-84d6-bbb0fdbe2638"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:33.515125 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.515083 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c42e891c-5dbf-4260-84d6-bbb0fdbe2638" (UID: "c42e891c-5dbf-4260-84d6-bbb0fdbe2638"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:20:33.515326 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.515248 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c42e891c-5dbf-4260-84d6-bbb0fdbe2638" (UID: "c42e891c-5dbf-4260-84d6-bbb0fdbe2638"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:33.515326 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.515321 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:33.515439 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.515340 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:33.515439 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.515353 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:33.515439 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.515368 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:33.515718 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.515692 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-data" (OuterVolumeSpecName: "istio-data") pod "c42e891c-5dbf-4260-84d6-bbb0fdbe2638" (UID: "c42e891c-5dbf-4260-84d6-bbb0fdbe2638"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:33.516930 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.516899 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c42e891c-5dbf-4260-84d6-bbb0fdbe2638" (UID: "c42e891c-5dbf-4260-84d6-bbb0fdbe2638"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:20:33.517076 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.517058 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c42e891c-5dbf-4260-84d6-bbb0fdbe2638" (UID: "c42e891c-5dbf-4260-84d6-bbb0fdbe2638"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:20:33.517167 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.517151 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-token" (OuterVolumeSpecName: "istio-token") pod "c42e891c-5dbf-4260-84d6-bbb0fdbe2638" (UID: "c42e891c-5dbf-4260-84d6-bbb0fdbe2638"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:33.517372 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.517354 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-kube-api-access-qgn7x" (OuterVolumeSpecName: "kube-api-access-qgn7x") pod "c42e891c-5dbf-4260-84d6-bbb0fdbe2638" (UID: "c42e891c-5dbf-4260-84d6-bbb0fdbe2638"). InnerVolumeSpecName "kube-api-access-qgn7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:20:33.616344 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.616310 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:33.616344 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.616338 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:33.616344 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.616347 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgn7x\" (UniqueName: \"kubernetes.io/projected/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-kube-api-access-qgn7x\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:33.616612 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.616356 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:33.616612 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:33.616364 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c42e891c-5dbf-4260-84d6-bbb0fdbe2638-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:20:34.407840 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:34.407809 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj" Jul 07 00:20:34.439286 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:34.439203 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj"] Jul 07 00:20:34.448717 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:34.448677 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-5b1463-openshift-default-c7ff89d67-vnzcj"] Jul 07 00:20:36.181763 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:36.181731 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" path="/var/lib/kubelet/pods/c42e891c-5dbf-4260-84d6-bbb0fdbe2638/volumes" Jul 07 00:20:37.427233 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.427201 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps"] Jul 07 00:20:37.427687 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.427672 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" Jul 07 00:20:37.427687 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.427688 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" Jul 07 00:20:37.427765 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.427707 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" Jul 07 00:20:37.427765 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.427714 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" Jul 07 00:20:37.427826 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.427787 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" Jul 07 00:20:37.427826 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.427794 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" Jul 07 00:20:37.427826 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.427802 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6367ff1-fae2-4641-b8b9-d540d4615de1" containerName="istio-proxy" Jul 07 00:20:37.432421 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.432396 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.434727 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.434694 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c6e67e-openshift-default-dockercfg-lljh5\"" Jul 07 00:20:37.442735 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.442706 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps"] Jul 07 00:20:37.553092 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.553055 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-data\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.553092 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.553101 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-envoy\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.553343 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.553170 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-socket\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.553343 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.553278 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621556df-25d9-4446-9036-ac91f89902d0-istiod-ca-cert\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.553343 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.553316 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-credential-socket\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.553459 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.553388 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-istio-token\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.553459 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.553418 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf8g7\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-kube-api-access-vf8g7\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.553578 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.553467 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-certs\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.553578 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.553536 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621556df-25d9-4446-9036-ac91f89902d0-istio-podinfo\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.654748 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.654714 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621556df-25d9-4446-9036-ac91f89902d0-istiod-ca-cert\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.654748 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.654751 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-credential-socket\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.654778 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-istio-token\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.654800 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf8g7\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-kube-api-access-vf8g7\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.654837 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-certs\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.654861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621556df-25d9-4446-9036-ac91f89902d0-istio-podinfo\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655016 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.654912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-data\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.655130 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-envoy\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.655171 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-socket\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.655225 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-data\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655451 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.655290 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-certs\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655536 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.655457 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-credential-socket\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655536 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.655509 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621556df-25d9-4446-9036-ac91f89902d0-istiod-ca-cert\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.655654 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.655576 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-socket\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.657264 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.657236 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-envoy\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.657408 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.657388 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621556df-25d9-4446-9036-ac91f89902d0-istio-podinfo\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.662198 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.662176 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-istio-token\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.662756 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.662736 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf8g7\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-kube-api-access-vf8g7\") pod \"e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.744735 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.744588 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:37.893998 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.893969 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps"] Jul 07 00:20:37.895264 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:20:37.895241 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621556df_25d9_4446_9036_ac91f89902d0.slice/crio-076e7d0ca59b21952089b5fd31b503c897b26c8125c2c7b9844e7d1bb1e67e70 WatchSource:0}: Error finding container 076e7d0ca59b21952089b5fd31b503c897b26c8125c2c7b9844e7d1bb1e67e70: Status 404 returned error can't find the container with id 076e7d0ca59b21952089b5fd31b503c897b26c8125c2c7b9844e7d1bb1e67e70 Jul 07 00:20:37.897572 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.897538 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:37.897674 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.897622 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:37.897674 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:37.897651 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:38.436256 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:38.435246 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" event={"ID":"621556df-25d9-4446-9036-ac91f89902d0","Type":"ContainerStarted","Data":"05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574"} Jul 07 00:20:38.436256 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:38.435312 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" event={"ID":"621556df-25d9-4446-9036-ac91f89902d0","Type":"ContainerStarted","Data":"076e7d0ca59b21952089b5fd31b503c897b26c8125c2c7b9844e7d1bb1e67e70"} Jul 07 00:20:38.460596 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:38.460540 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" podStartSLOduration=1.460519445 podStartE2EDuration="1.460519445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:20:38.456814912 +0000 UTC m=+1948.719487109" watchObservedRunningTime="2026-07-07 00:20:38.460519445 +0000 UTC m=+1948.723191631" Jul 07 00:20:38.745688 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:38.745650 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:38.755456 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:38.755405 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:39.445394 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:39.445357 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:39.451765 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:39.451729 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:41.603573 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:41.603531 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:20:45.491032 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:45.491004 2583 generic.go:356] "Generic (PLEG): container finished" podID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerID="7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9" exitCode=0 Jul 07 00:20:45.491388 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:45.491073 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" event={"ID":"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7","Type":"ContainerDied","Data":"7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9"} Jul 07 00:20:45.491488 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:45.491460 2583 scope.go:117] "RemoveContainer" containerID="7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9" Jul 07 00:20:45.492609 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:45.492584 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:45.492666 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:45.492652 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:45.492705 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:45.492688 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:46.378012 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:46.377984 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:46.506897 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:46.506859 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" event={"ID":"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7","Type":"ContainerStarted","Data":"92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba"} Jul 07 00:20:47.383462 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:47.383419 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:47.384770 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:47.384732 2583 patch_prober.go:28] interesting pod/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 07 00:20:47.384923 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:47.384819 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 07 00:20:48.383466 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:48.383422 2583 patch_prober.go:28] interesting pod/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 07 00:20:48.384003 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:48.383520 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 07 00:20:49.383437 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:49.383399 2583 patch_prober.go:28] interesting pod/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 07 00:20:49.383694 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:49.383462 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 07 00:20:50.209578 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:50.209550 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:20:50.382973 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:50.382938 2583 patch_prober.go:28] interesting pod/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 07 00:20:50.383192 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:50.383023 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 07 00:20:51.387022 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:51.386993 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:51.387523 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:51.387464 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:51.388150 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:51.388132 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:20:54.447085 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:54.447045 2583 patch_prober.go:28] interesting pod/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 07 00:20:54.447553 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:54.447118 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 07 00:20:54.565281 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:54.565244 2583 generic.go:356] "Generic (PLEG): container finished" podID="621556df-25d9-4446-9036-ac91f89902d0" containerID="05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574" exitCode=0 Jul 07 00:20:54.565465 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:54.565318 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" event={"ID":"621556df-25d9-4446-9036-ac91f89902d0","Type":"ContainerDied","Data":"05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574"} Jul 07 00:20:54.565786 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:54.565767 2583 scope.go:117] "RemoveContainer" containerID="05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574" Jul 07 00:20:54.566818 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:54.566781 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:54.566911 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:54.566857 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:54.566911 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:54.566890 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 00:20:55.573396 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:55.573362 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" event={"ID":"621556df-25d9-4446-9036-ac91f89902d0","Type":"ContainerStarted","Data":"b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8"} Jul 07 00:20:55.745570 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:55.745527 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:55.745887 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:55.745853 2583 patch_prober.go:28] interesting pod/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 07 00:20:55.745974 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:55.745925 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 07 00:20:56.745491 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:56.745434 2583 patch_prober.go:28] interesting pod/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 07 00:20:56.746132 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:56.746096 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 07 00:20:57.745774 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:57.745720 2583 patch_prober.go:28] interesting pod/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 07 00:20:57.746273 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:57.745795 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 07 00:20:58.745324 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:58.745282 2583 patch_prober.go:28] interesting pod/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body= Jul 07 00:20:58.745663 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:58.745624 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" Jul 07 00:20:59.749115 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:59.749082 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:59.749599 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:59.749539 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:20:59.750389 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:20:59.750366 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:21:07.802183 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:21:07.802095 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:21:29.390413 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:21:29.390371 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:21:42.208643 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:21:42.208605 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:21:59.609635 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:21:59.609599 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:22:10.663063 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:22:10.663028 2583 scope.go:117] "RemoveContainer" containerID="0fbd80bb72af210df6724a1fa4ba5732ac3ce2104670f5e8e1a8ab071e0e89c7" Jul 07 00:22:15.394560 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:22:15.394524 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:22:33.001895 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:22:33.001809 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:22:45.923202 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:22:45.923166 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:23:03.230056 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:23:03.230020 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:23:18.407961 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:23:18.407921 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:23:36.103598 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:23:36.103555 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:23:47.903030 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:23:47.902989 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:24:06.904296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:24:06.904199 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:24:40.504924 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:24:40.504893 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:24:57.294517 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:24:57.294461 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:25:14.093181 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:25:14.093134 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:25:27.397404 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:25:27.397373 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:25:52.096573 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:25:52.096538 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:26:05.206147 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:26:05.206099 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:26:19.704059 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:26:19.704022 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:26:32.700273 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:26:32.700228 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:27:10.720375 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:10.720294 2583 scope.go:117] "RemoveContainer" containerID="3a55c83ebb49c2d72a6753b5e39068c59dbdd7f58c87dcc9e4a3585f10b82bda" Jul 07 00:27:11.133420 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:11.133388 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77"] Jul 07 00:27:11.133653 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:11.133624 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" containerID="cri-o://92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba" gracePeriod=30 Jul 07 00:27:14.997965 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:14.997926 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-d64r9"] Jul 07 00:27:16.392200 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.392174 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:27:16.436087 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436007 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-envoy\") pod \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " Jul 07 00:27:16.436087 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436042 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-credential-socket\") pod \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " Jul 07 00:27:16.436306 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436100 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-data\") pod \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " Jul 07 00:27:16.436306 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436134 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-certs\") pod \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " Jul 07 00:27:16.436306 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436191 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-token\") pod \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " Jul 07 00:27:16.436306 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436215 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-socket\") pod \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " Jul 07 00:27:16.436306 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436247 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64snj\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-kube-api-access-64snj\") pod \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " Jul 07 00:27:16.436306 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436277 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-podinfo\") pod \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " Jul 07 00:27:16.436306 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436304 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istiod-ca-cert\") pod \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\" (UID: \"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7\") " Jul 07 00:27:16.436709 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436564 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" (UID: "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:16.436709 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436693 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:16.436825 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436789 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" (UID: "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:16.436897 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.436874 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" (UID: "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:27:16.437137 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.437102 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" (UID: "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:16.437490 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.437449 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-data" (OuterVolumeSpecName: "istio-data") pod "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" (UID: "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:16.438490 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.438449 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" (UID: "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:16.438915 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.438894 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" (UID: "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:27:16.439008 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.438916 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-token" (OuterVolumeSpecName: "istio-token") pod "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" (UID: "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:27:16.439008 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.438958 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-kube-api-access-64snj" (OuterVolumeSpecName: "kube-api-access-64snj") pod "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" (UID: "780b0248-b6e4-4203-b2d1-e4f9be3c9dc7"). InnerVolumeSpecName "kube-api-access-64snj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:27:16.537250 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.537215 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:16.537250 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.537245 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:16.537250 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.537256 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:16.537533 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.537264 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:16.537533 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.537272 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:16.537533 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.537281 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64snj\" (UniqueName: \"kubernetes.io/projected/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-kube-api-access-64snj\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:16.537533 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.537288 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:16.537533 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.537297 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:16.868570 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.868539 2583 generic.go:356] "Generic (PLEG): container finished" podID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerID="92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba" exitCode=0 Jul 07 00:27:16.868751 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.868586 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" Jul 07 00:27:16.868751 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.868584 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" event={"ID":"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7","Type":"ContainerDied","Data":"92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba"} Jul 07 00:27:16.868751 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.868703 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77" event={"ID":"780b0248-b6e4-4203-b2d1-e4f9be3c9dc7","Type":"ContainerDied","Data":"55485e043bf0fb18b889772201f15bfa4d7c3cc822da86e138df993b25d1270e"} Jul 07 00:27:16.868751 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.868731 2583 scope.go:117] "RemoveContainer" containerID="92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba" Jul 07 00:27:16.879104 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.879083 2583 scope.go:117] "RemoveContainer" containerID="7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9" Jul 07 00:27:16.887219 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.887203 2583 scope.go:117] "RemoveContainer" containerID="92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba" Jul 07 00:27:16.887441 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:27:16.887422 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba\": container with ID starting with 92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba not found: ID does not exist" containerID="92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba" Jul 07 00:27:16.887515 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.887452 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba"} err="failed to get container status \"92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba\": rpc error: code = NotFound desc = could not find container \"92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba\": container with ID starting with 92652959f015a52f84bdecf4eee3fc6e3d636791d0d7dc6924ffe3e46f30d8ba not found: ID does not exist" Jul 07 00:27:16.887515 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.887485 2583 scope.go:117] "RemoveContainer" containerID="7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9" Jul 07 00:27:16.887712 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:27:16.887697 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9\": container with ID starting with 7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9 not found: ID does not exist" containerID="7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9" Jul 07 00:27:16.887758 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.887718 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9"} err="failed to get container status \"7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9\": rpc error: code = NotFound desc = could not find container \"7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9\": container with ID starting with 7cb6929c01ebf6a555314967a5fed2343328d4e31403ce6b3f2b43f8c0de21e9 not found: ID does not exist" Jul 07 00:27:16.892708 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.892685 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77"] Jul 07 00:27:16.898820 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:16.898801 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-b60cd9-openshift-default-66bbdfbdcf-v5h77"] Jul 07 00:27:18.183244 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:18.183202 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" path="/var/lib/kubelet/pods/780b0248-b6e4-4203-b2d1-e4f9be3c9dc7/volumes" Jul 07 00:27:18.358307 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:18.358271 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps"] Jul 07 00:27:18.358554 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:18.358510 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" containerID="cri-o://b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8" gracePeriod=30 Jul 07 00:27:18.999710 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:18.999667 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-785697d47c-j9vjc_2445a1d8-2828-4deb-96e4-1f0004c17837/authorino/0.log" Jul 07 00:27:23.401662 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.401626 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5w9s5_cc8e98a6-58cf-4d7e-a9ee-090c8939a6db/manager/0.log" Jul 07 00:27:23.518910 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.518876 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hghqm_7682d489-0d6c-4b87-bdba-0cbcde4d6868/manager/0.log" Jul 07 00:27:23.616162 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.616137 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:27:23.711340 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711241 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vf8g7\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-kube-api-access-vf8g7\") pod \"621556df-25d9-4446-9036-ac91f89902d0\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " Jul 07 00:27:23.711340 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711285 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621556df-25d9-4446-9036-ac91f89902d0-istio-podinfo\") pod \"621556df-25d9-4446-9036-ac91f89902d0\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " Jul 07 00:27:23.711340 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711325 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-certs\") pod \"621556df-25d9-4446-9036-ac91f89902d0\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " Jul 07 00:27:23.711680 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711367 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-data\") pod \"621556df-25d9-4446-9036-ac91f89902d0\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " Jul 07 00:27:23.711680 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711391 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-istio-token\") pod \"621556df-25d9-4446-9036-ac91f89902d0\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " Jul 07 00:27:23.711680 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711416 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621556df-25d9-4446-9036-ac91f89902d0-istiod-ca-cert\") pod \"621556df-25d9-4446-9036-ac91f89902d0\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " Jul 07 00:27:23.711680 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711431 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-credential-socket\") pod \"621556df-25d9-4446-9036-ac91f89902d0\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " Jul 07 00:27:23.711680 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711460 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-socket\") pod \"621556df-25d9-4446-9036-ac91f89902d0\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " Jul 07 00:27:23.711680 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711526 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-envoy\") pod \"621556df-25d9-4446-9036-ac91f89902d0\" (UID: \"621556df-25d9-4446-9036-ac91f89902d0\") " Jul 07 00:27:23.711987 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711716 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "621556df-25d9-4446-9036-ac91f89902d0" (UID: "621556df-25d9-4446-9036-ac91f89902d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:23.711987 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711926 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "621556df-25d9-4446-9036-ac91f89902d0" (UID: "621556df-25d9-4446-9036-ac91f89902d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:23.711987 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711962 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/621556df-25d9-4446-9036-ac91f89902d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "621556df-25d9-4446-9036-ac91f89902d0" (UID: "621556df-25d9-4446-9036-ac91f89902d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 00:27:23.712148 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.711978 2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-certs\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:23.712148 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.712084 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "621556df-25d9-4446-9036-ac91f89902d0" (UID: "621556df-25d9-4446-9036-ac91f89902d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:23.712347 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.712326 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "621556df-25d9-4446-9036-ac91f89902d0" (UID: "621556df-25d9-4446-9036-ac91f89902d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:23.713626 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.713604 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-kube-api-access-vf8g7" (OuterVolumeSpecName: "kube-api-access-vf8g7") pod "621556df-25d9-4446-9036-ac91f89902d0" (UID: "621556df-25d9-4446-9036-ac91f89902d0"). InnerVolumeSpecName "kube-api-access-vf8g7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:27:23.713794 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.713775 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "621556df-25d9-4446-9036-ac91f89902d0" (UID: "621556df-25d9-4446-9036-ac91f89902d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 00:27:23.714006 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.713989 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "621556df-25d9-4446-9036-ac91f89902d0" (UID: "621556df-25d9-4446-9036-ac91f89902d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 00:27:23.714077 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.714064 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/621556df-25d9-4446-9036-ac91f89902d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "621556df-25d9-4446-9036-ac91f89902d0" (UID: "621556df-25d9-4446-9036-ac91f89902d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 00:27:23.759019 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.758985 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723040-njdwh_4a50d927-75ad-49c3-a48a-f72ff3f6027f/cleanup/0.log" Jul 07 00:27:23.813153 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.813112 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vf8g7\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-kube-api-access-vf8g7\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:23.813153 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.813147 2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/621556df-25d9-4446-9036-ac91f89902d0-istio-podinfo\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:23.813367 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.813162 2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-data\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:23.813367 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.813175 2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/621556df-25d9-4446-9036-ac91f89902d0-istio-token\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:23.813367 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.813186 2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/621556df-25d9-4446-9036-ac91f89902d0-istiod-ca-cert\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:23.813367 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.813197 2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-credential-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:23.813367 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.813208 2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-workload-socket\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:23.813367 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.813219 2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/621556df-25d9-4446-9036-ac91f89902d0-istio-envoy\") on node \"ip-10-0-137-188.ec2.internal\" DevicePath \"\"" Jul 07 00:27:23.862029 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.861993 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723055-mmnms_54cc54a8-c744-462a-99d3-a9b43a4de909/cleanup/0.log" Jul 07 00:27:23.913337 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.913303 2583 generic.go:356] "Generic (PLEG): container finished" podID="621556df-25d9-4446-9036-ac91f89902d0" containerID="b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8" exitCode=0 Jul 07 00:27:23.913525 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.913387 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" Jul 07 00:27:23.913525 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.913392 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" event={"ID":"621556df-25d9-4446-9036-ac91f89902d0","Type":"ContainerDied","Data":"b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8"} Jul 07 00:27:23.913525 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.913504 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps" event={"ID":"621556df-25d9-4446-9036-ac91f89902d0","Type":"ContainerDied","Data":"076e7d0ca59b21952089b5fd31b503c897b26c8125c2c7b9844e7d1bb1e67e70"} Jul 07 00:27:23.913720 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.913530 2583 scope.go:117] "RemoveContainer" containerID="b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8" Jul 07 00:27:23.923403 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.923384 2583 scope.go:117] "RemoveContainer" containerID="05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574" Jul 07 00:27:23.931938 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.931920 2583 scope.go:117] "RemoveContainer" containerID="b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8" Jul 07 00:27:23.932188 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:27:23.932169 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8\": container with ID starting with b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8 not found: ID does not exist" containerID="b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8" Jul 07 00:27:23.932233 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.932197 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8"} err="failed to get container status \"b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8\": rpc error: code = NotFound desc = could not find container \"b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8\": container with ID starting with b41eb881b1bc414b25e8bfa0c64af211163cf9a420c88e0d8ce86b230361b5e8 not found: ID does not exist" Jul 07 00:27:23.932233 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.932216 2583 scope.go:117] "RemoveContainer" containerID="05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574" Jul 07 00:27:23.932462 ip-10-0-137-188 kubenswrapper[2583]: E0707 00:27:23.932447 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574\": container with ID starting with 05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574 not found: ID does not exist" containerID="05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574" Jul 07 00:27:23.932519 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.932482 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574"} err="failed to get container status \"05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574\": rpc error: code = NotFound desc = could not find container \"05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574\": container with ID starting with 05de0bc17f747544b753bd210b27420b79d0c5768eeccc738109ce8be567e574 not found: ID does not exist" Jul 07 00:27:23.936355 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.936321 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps"] Jul 07 00:27:23.940466 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:23.940439 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c6e67e-openshift-default-786f4d79c9-z9lps"] Jul 07 00:27:24.099261 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:24.099229 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-b456b_8414f226-13c8-429a-8a7c-c77150cc7cb1/server/0.log" Jul 07 00:27:24.182201 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:24.182159 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="621556df-25d9-4446-9036-ac91f89902d0" path="/var/lib/kubelet/pods/621556df-25d9-4446-9036-ac91f89902d0/volumes" Jul 07 00:27:24.214681 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:24.214449 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tp54k_59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d/manager/0.log" Jul 07 00:27:24.457972 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:24.457884 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4d578756-r2dx8_cbe40b23-970e-4072-a3a6-78985342407b/manager/0.log" Jul 07 00:27:24.475243 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:24.475209 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4d578756-r2dx8_cbe40b23-970e-4072-a3a6-78985342407b/copy-manifests/0.log" Jul 07 00:27:25.458848 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.458814 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh_56f2771f-27ec-46f3-ae8c-67172913f5aa/util/0.log" Jul 07 00:27:25.470995 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.470967 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh_56f2771f-27ec-46f3-ae8c-67172913f5aa/pull/0.log" Jul 07 00:27:25.480741 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.480717 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh_56f2771f-27ec-46f3-ae8c-67172913f5aa/extract/0.log" Jul 07 00:27:25.593697 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.593671 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v_edccff2e-a93c-4bb4-afcf-0b99dfb66e7f/util/0.log" Jul 07 00:27:25.601203 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.601179 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v_edccff2e-a93c-4bb4-afcf-0b99dfb66e7f/pull/0.log" Jul 07 00:27:25.609747 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.609723 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v_edccff2e-a93c-4bb4-afcf-0b99dfb66e7f/extract/0.log" Jul 07 00:27:25.722845 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.722752 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477_d0bd98e0-34ef-47e3-8375-1a13e57a7789/util/0.log" Jul 07 00:27:25.734789 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.734764 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477_d0bd98e0-34ef-47e3-8375-1a13e57a7789/pull/0.log" Jul 07 00:27:25.748685 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.748663 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477_d0bd98e0-34ef-47e3-8375-1a13e57a7789/extract/0.log" Jul 07 00:27:25.857850 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.857824 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg_d41a7224-0d15-4901-8657-b622da3d11cc/util/0.log" Jul 07 00:27:25.869092 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.869057 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg_d41a7224-0d15-4901-8657-b622da3d11cc/pull/0.log" Jul 07 00:27:25.880790 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:25.880755 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg_d41a7224-0d15-4901-8657-b622da3d11cc/extract/0.log" Jul 07 00:27:26.000290 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:26.000251 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-785697d47c-j9vjc_2445a1d8-2828-4deb-96e4-1f0004c17837/authorino/0.log" Jul 07 00:27:26.118436 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:26.118402 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-npdkl_a147ff07-4edc-434e-abfe-c0e8bcfa1452/manager/0.log" Jul 07 00:27:26.231277 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:26.231249 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-h542l_e3eb50fe-c234-4bbb-8b86-708e4b82430b/manager/0.log" Jul 07 00:27:26.342933 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:26.342855 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-62tvv_975e7a09-fc30-4d58-a83a-dc98a208b07e/kuadrant-console-plugin/0.log" Jul 07 00:27:26.458347 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:26.458321 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-9ppbd_1fd4387f-189a-4d28-8d78-3bb6f43b8381/registry-server/0.log" Jul 07 00:27:26.701405 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:26.701335 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-d64r9_800bf725-b7a9-4b66-ac9c-bd9a13324f06/limitador/0.log" Jul 07 00:27:26.818910 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:26.818882 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-s6644_13435547-2d5c-4324-9a31-95d18fb2f6ee/manager/0.log" Jul 07 00:27:27.172049 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:27.172020 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t24l7_a24e7fdb-20db-4a53-83e8-caf0225145d1/istio-proxy/0.log" Jul 07 00:27:27.500920 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:27.500894 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4984cb9-8fgfb_ece0863e-d2c2-4310-b949-0bcc0e3279b0/kube-auth-proxy/0.log" Jul 07 00:27:27.614523 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:27.614492 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cbxpr_e2eb3722-5915-4084-b152-86187682a2d3/istio-proxy/1.log" Jul 07 00:27:33.150917 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.150882 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s7kft/must-gather-6nhsd"] Jul 07 00:27:33.151304 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151285 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" Jul 07 00:27:33.151304 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151296 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" Jul 07 00:27:33.151383 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151310 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" Jul 07 00:27:33.151383 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151315 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" Jul 07 00:27:33.151383 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151323 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" Jul 07 00:27:33.151383 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151329 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="c42e891c-5dbf-4260-84d6-bbb0fdbe2638" containerName="istio-proxy" Jul 07 00:27:33.151383 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151339 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" Jul 07 00:27:33.151383 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151345 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" Jul 07 00:27:33.151598 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151409 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" Jul 07 00:27:33.151598 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151417 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" Jul 07 00:27:33.151598 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151423 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" Jul 07 00:27:33.151598 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151511 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" Jul 07 00:27:33.151598 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151517 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="621556df-25d9-4446-9036-ac91f89902d0" containerName="istio-proxy" Jul 07 00:27:33.151749 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.151600 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="780b0248-b6e4-4203-b2d1-e4f9be3c9dc7" containerName="istio-proxy" Jul 07 00:27:33.154612 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.154593 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s7kft/must-gather-6nhsd" Jul 07 00:27:33.156866 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.156846 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s7kft\"/\"kube-root-ca.crt\"" Jul 07 00:27:33.157347 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.157329 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s7kft\"/\"openshift-service-ca.crt\"" Jul 07 00:27:33.157620 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.157596 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s7kft\"/\"default-dockercfg-grvfn\"" Jul 07 00:27:33.163663 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.163641 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s7kft/must-gather-6nhsd"] Jul 07 00:27:33.300458 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.300420 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9a5d258c-2723-480b-8c80-20708bbbf12b-must-gather-output\") pod \"must-gather-6nhsd\" (UID: \"9a5d258c-2723-480b-8c80-20708bbbf12b\") " pod="openshift-must-gather-s7kft/must-gather-6nhsd" Jul 07 00:27:33.300686 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.300497 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26b8b\" (UniqueName: \"kubernetes.io/projected/9a5d258c-2723-480b-8c80-20708bbbf12b-kube-api-access-26b8b\") pod \"must-gather-6nhsd\" (UID: \"9a5d258c-2723-480b-8c80-20708bbbf12b\") " pod="openshift-must-gather-s7kft/must-gather-6nhsd" Jul 07 00:27:33.401123 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.401033 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9a5d258c-2723-480b-8c80-20708bbbf12b-must-gather-output\") pod \"must-gather-6nhsd\" (UID: \"9a5d258c-2723-480b-8c80-20708bbbf12b\") " pod="openshift-must-gather-s7kft/must-gather-6nhsd" Jul 07 00:27:33.401123 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.401086 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26b8b\" (UniqueName: \"kubernetes.io/projected/9a5d258c-2723-480b-8c80-20708bbbf12b-kube-api-access-26b8b\") pod \"must-gather-6nhsd\" (UID: \"9a5d258c-2723-480b-8c80-20708bbbf12b\") " pod="openshift-must-gather-s7kft/must-gather-6nhsd" Jul 07 00:27:33.401367 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.401348 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/9a5d258c-2723-480b-8c80-20708bbbf12b-must-gather-output\") pod \"must-gather-6nhsd\" (UID: \"9a5d258c-2723-480b-8c80-20708bbbf12b\") " pod="openshift-must-gather-s7kft/must-gather-6nhsd" Jul 07 00:27:33.409007 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.408983 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26b8b\" (UniqueName: \"kubernetes.io/projected/9a5d258c-2723-480b-8c80-20708bbbf12b-kube-api-access-26b8b\") pod \"must-gather-6nhsd\" (UID: \"9a5d258c-2723-480b-8c80-20708bbbf12b\") " pod="openshift-must-gather-s7kft/must-gather-6nhsd" Jul 07 00:27:33.464400 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.464373 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s7kft/must-gather-6nhsd" Jul 07 00:27:33.605315 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.605286 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s7kft/must-gather-6nhsd"] Jul 07 00:27:33.605758 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:27:33.605734 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a5d258c_2723_480b_8c80_20708bbbf12b.slice/crio-3f3a39d9c20fb14f47eb8bbf61d1ff35663d9e3d5b6bfa19a317d51eca57ce7a WatchSource:0}: Error finding container 3f3a39d9c20fb14f47eb8bbf61d1ff35663d9e3d5b6bfa19a317d51eca57ce7a: Status 404 returned error can't find the container with id 3f3a39d9c20fb14f47eb8bbf61d1ff35663d9e3d5b6bfa19a317d51eca57ce7a Jul 07 00:27:33.607776 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.607760 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 00:27:33.973360 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:33.973330 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s7kft/must-gather-6nhsd" event={"ID":"9a5d258c-2723-480b-8c80-20708bbbf12b","Type":"ContainerStarted","Data":"3f3a39d9c20fb14f47eb8bbf61d1ff35663d9e3d5b6bfa19a317d51eca57ce7a"} Jul 07 00:27:34.985186 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:34.985143 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s7kft/must-gather-6nhsd" event={"ID":"9a5d258c-2723-480b-8c80-20708bbbf12b","Type":"ContainerStarted","Data":"ac223a3b54c50e95da5fc0ac87b4c31a5799b924dc82683777168f3503f94310"} Jul 07 00:27:34.985186 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:34.985199 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s7kft/must-gather-6nhsd" event={"ID":"9a5d258c-2723-480b-8c80-20708bbbf12b","Type":"ContainerStarted","Data":"851df728821c37c9cea560557f1216810563c205a104f462af1f5a1cbdc6c342"} Jul 07 00:27:35.000826 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:35.000765 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s7kft/must-gather-6nhsd" podStartSLOduration=1.250224536 podStartE2EDuration="2.00074498s" totalImagesPullingTime="750.520444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 00:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:27:34.998993414 +0000 UTC m=+2365.261665610" watchObservedRunningTime="2026-07-07 00:27:35.00074498 +0000 UTC m=+2365.263417176" Jul 07 00:27:36.003692 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:36.003662 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vcbhn_9d9114c9-cf23-440f-a966-794fe592ced5/global-pull-secret-syncer/0.log" Jul 07 00:27:36.083935 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:36.083901 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8t9lx_145be0a6-160e-481e-987c-fb1af73e4257/konnectivity-agent/0.log" Jul 07 00:27:36.193574 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:36.193538 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-188.ec2.internal_8ad2d9a7286370db5ed9312271a4a122/haproxy/0.log" Jul 07 00:27:40.696045 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.696007 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh_56f2771f-27ec-46f3-ae8c-67172913f5aa/extract/0.log" Jul 07 00:27:40.727396 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.727358 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh_56f2771f-27ec-46f3-ae8c-67172913f5aa/util/0.log" Jul 07 00:27:40.757216 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.757153 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k54gh_56f2771f-27ec-46f3-ae8c-67172913f5aa/pull/0.log" Jul 07 00:27:40.788434 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.788408 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v_edccff2e-a93c-4bb4-afcf-0b99dfb66e7f/extract/0.log" Jul 07 00:27:40.814351 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.814326 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v_edccff2e-a93c-4bb4-afcf-0b99dfb66e7f/util/0.log" Jul 07 00:27:40.839154 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.839042 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xfn2v_edccff2e-a93c-4bb4-afcf-0b99dfb66e7f/pull/0.log" Jul 07 00:27:40.868516 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.868462 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477_d0bd98e0-34ef-47e3-8375-1a13e57a7789/extract/0.log" Jul 07 00:27:40.892790 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.892758 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477_d0bd98e0-34ef-47e3-8375-1a13e57a7789/util/0.log" Jul 07 00:27:40.942372 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.942321 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73t4477_d0bd98e0-34ef-47e3-8375-1a13e57a7789/pull/0.log" Jul 07 00:27:40.971682 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.970006 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg_d41a7224-0d15-4901-8657-b622da3d11cc/extract/0.log" Jul 07 00:27:40.994404 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:40.994370 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg_d41a7224-0d15-4901-8657-b622da3d11cc/util/0.log" Jul 07 00:27:41.017391 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:41.017356 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1h28bg_d41a7224-0d15-4901-8657-b622da3d11cc/pull/0.log" Jul 07 00:27:41.359497 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:41.357409 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-785697d47c-j9vjc_2445a1d8-2828-4deb-96e4-1f0004c17837/authorino/0.log" Jul 07 00:27:41.388215 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:41.388185 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-npdkl_a147ff07-4edc-434e-abfe-c0e8bcfa1452/manager/0.log" Jul 07 00:27:41.424390 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:41.424263 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-h542l_e3eb50fe-c234-4bbb-8b86-708e4b82430b/manager/0.log" Jul 07 00:27:41.454963 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:41.454927 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-62tvv_975e7a09-fc30-4d58-a83a-dc98a208b07e/kuadrant-console-plugin/0.log" Jul 07 00:27:41.500941 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:41.500900 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-9ppbd_1fd4387f-189a-4d28-8d78-3bb6f43b8381/registry-server/0.log" Jul 07 00:27:41.708417 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:41.708333 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-d64r9_800bf725-b7a9-4b66-ac9c-bd9a13324f06/limitador/0.log" Jul 07 00:27:41.788784 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:41.788744 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-s6644_13435547-2d5c-4324-9a31-95d18fb2f6ee/manager/0.log" Jul 07 00:27:43.290521 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.290486 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_719b2d13-e001-4fde-8c72-6d082d75be25/alertmanager/0.log" Jul 07 00:27:43.318643 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.318559 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_719b2d13-e001-4fde-8c72-6d082d75be25/config-reloader/0.log" Jul 07 00:27:43.345840 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.345808 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_719b2d13-e001-4fde-8c72-6d082d75be25/kube-rbac-proxy-web/0.log" Jul 07 00:27:43.376785 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.376754 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_719b2d13-e001-4fde-8c72-6d082d75be25/kube-rbac-proxy/0.log" Jul 07 00:27:43.402056 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.402021 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_719b2d13-e001-4fde-8c72-6d082d75be25/kube-rbac-proxy-metric/0.log" Jul 07 00:27:43.427514 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.427467 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_719b2d13-e001-4fde-8c72-6d082d75be25/prom-label-proxy/0.log" Jul 07 00:27:43.457955 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.455348 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_719b2d13-e001-4fde-8c72-6d082d75be25/init-config-reloader/0.log" Jul 07 00:27:43.652670 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.652597 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-ghb2n_c92b6d08-724e-41aa-bafb-4f0b9fbe5628/monitoring-plugin/0.log" Jul 07 00:27:43.704445 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.704314 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5l7sg_fefc3ede-6c89-468a-80e4-e03f1f9fc913/node-exporter/0.log" Jul 07 00:27:43.730465 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.730432 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5l7sg_fefc3ede-6c89-468a-80e4-e03f1f9fc913/kube-rbac-proxy/0.log" Jul 07 00:27:43.759180 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.759081 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5l7sg_fefc3ede-6c89-468a-80e4-e03f1f9fc913/init-textfile/0.log" Jul 07 00:27:43.941306 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.941113 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-bxlbs_5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f/kube-rbac-proxy-main/0.log" Jul 07 00:27:43.968925 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.968897 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-bxlbs_5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f/kube-rbac-proxy-self/0.log" Jul 07 00:27:43.999029 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:43.999001 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-bxlbs_5fdd4fde-44da-40d4-b4a4-5ff1f25f2a0f/openshift-state-metrics/0.log" Jul 07 00:27:44.256144 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.256111 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-4p46b_243e7b42-f551-47ad-a169-c0fc8df2ca2d/prometheus-operator-admission-webhook/0.log" Jul 07 00:27:44.286326 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.286301 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cfd57dc85-qlqqh_19879c39-6c1e-4cdf-b480-a3c004613a96/telemeter-client/0.log" Jul 07 00:27:44.315958 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.315932 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cfd57dc85-qlqqh_19879c39-6c1e-4cdf-b480-a3c004613a96/reload/0.log" Jul 07 00:27:44.342413 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.342338 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7cfd57dc85-qlqqh_19879c39-6c1e-4cdf-b480-a3c004613a96/kube-rbac-proxy/0.log" Jul 07 00:27:44.381701 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.381675 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f56c4868-62rgg_94654697-ef15-4f75-b0b4-3a3f9023e3a0/thanos-query/0.log" Jul 07 00:27:44.405284 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.405255 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f56c4868-62rgg_94654697-ef15-4f75-b0b4-3a3f9023e3a0/kube-rbac-proxy-web/0.log" Jul 07 00:27:44.433449 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.433410 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f56c4868-62rgg_94654697-ef15-4f75-b0b4-3a3f9023e3a0/kube-rbac-proxy/0.log" Jul 07 00:27:44.457585 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.457555 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f56c4868-62rgg_94654697-ef15-4f75-b0b4-3a3f9023e3a0/prom-label-proxy/0.log" Jul 07 00:27:44.480726 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.480693 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f56c4868-62rgg_94654697-ef15-4f75-b0b4-3a3f9023e3a0/kube-rbac-proxy-rules/0.log" Jul 07 00:27:44.506868 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:44.506782 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5f56c4868-62rgg_94654697-ef15-4f75-b0b4-3a3f9023e3a0/kube-rbac-proxy-metrics/0.log" Jul 07 00:27:45.915004 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:45.914965 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j"] Jul 07 00:27:45.920725 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:45.920698 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:45.928638 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:45.928610 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j"] Jul 07 00:27:45.957296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:45.956157 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-podres\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:45.957296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:45.956226 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngp6j\" (UniqueName: \"kubernetes.io/projected/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-kube-api-access-ngp6j\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:45.957296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:45.956281 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-lib-modules\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:45.957296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:45.956320 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-sys\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:45.957296 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:45.956387 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-proc\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.057506 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.057437 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-proc\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.057718 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.057545 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-podres\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.057718 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.057575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngp6j\" (UniqueName: \"kubernetes.io/projected/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-kube-api-access-ngp6j\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.057718 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.057614 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-lib-modules\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.057718 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.057646 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-sys\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.057944 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.057727 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-sys\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.057944 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.057731 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-podres\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.057944 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.057610 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-proc\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.057944 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.057826 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-lib-modules\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.074038 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.073997 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngp6j\" (UniqueName: \"kubernetes.io/projected/96dd7c36-37dd-43cc-80a7-74358b8bbe8a-kube-api-access-ngp6j\") pod \"perf-node-gather-daemonset-ww55j\" (UID: \"96dd7c36-37dd-43cc-80a7-74358b8bbe8a\") " pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.235202 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.235122 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:46.430310 ip-10-0-137-188 kubenswrapper[2583]: W0707 00:27:46.430219 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod96dd7c36_37dd_43cc_80a7_74358b8bbe8a.slice/crio-9a5ab0e0e2c96e47baf9463a267159b55112f8486f8bdd076416e06171195f6e WatchSource:0}: Error finding container 9a5ab0e0e2c96e47baf9463a267159b55112f8486f8bdd076416e06171195f6e: Status 404 returned error can't find the container with id 9a5ab0e0e2c96e47baf9463a267159b55112f8486f8bdd076416e06171195f6e Jul 07 00:27:46.430582 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:46.430453 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j"] Jul 07 00:27:47.000314 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:47.000281 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-42rh9_05df0493-f960-4a6a-b620-e532c61e18de/download-server/0.log" Jul 07 00:27:47.096458 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:47.096419 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" event={"ID":"96dd7c36-37dd-43cc-80a7-74358b8bbe8a","Type":"ContainerStarted","Data":"45b67ccc84d82e5d9851c85a41915c0e59e3370008f84704d6136d3491558298"} Jul 07 00:27:47.096668 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:47.096506 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" event={"ID":"96dd7c36-37dd-43cc-80a7-74358b8bbe8a","Type":"ContainerStarted","Data":"9a5ab0e0e2c96e47baf9463a267159b55112f8486f8bdd076416e06171195f6e"} Jul 07 00:27:47.096668 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:47.096547 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:48.641361 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:48.641328 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bs4cn_41c2146f-78d2-499c-81f3-ae76240f8c62/dns/0.log" Jul 07 00:27:48.663917 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:48.663888 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bs4cn_41c2146f-78d2-499c-81f3-ae76240f8c62/kube-rbac-proxy/0.log" Jul 07 00:27:48.828528 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:48.828500 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sm2dg_673c0dc9-eae9-4d18-ac43-825856e88954/dns-node-resolver/0.log" Jul 07 00:27:49.331796 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:49.331762 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29723040-q5fxn_3a41f65f-ca12-4745-8d9f-0c86e5e081cb/image-pruner/0.log" Jul 07 00:27:49.375399 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:49.375373 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-755c4fcb8f-pwv98_0350166e-a62f-49f8-911b-9f6f7870a562/registry/0.log" Jul 07 00:27:49.445653 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:49.445626 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j9bq7_d751b2b6-b78f-41b3-b895-5d559e9926d1/node-ca/0.log" Jul 07 00:27:50.443236 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:50.443199 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t24l7_a24e7fdb-20db-4a53-83e8-caf0225145d1/istio-proxy/0.log" Jul 07 00:27:50.600123 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:50.600093 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4984cb9-8fgfb_ece0863e-d2c2-4310-b949-0bcc0e3279b0/kube-auth-proxy/0.log" Jul 07 00:27:50.638834 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:50.638805 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cbxpr_e2eb3722-5915-4084-b152-86187682a2d3/istio-proxy/0.log" Jul 07 00:27:50.679174 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:50.679145 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cbxpr_e2eb3722-5915-4084-b152-86187682a2d3/istio-proxy/1.log" Jul 07 00:27:51.486015 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:51.485990 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5pg5w_0812878f-d9b5-4456-8b62-9a020d4cc2a4/serve-healthcheck-canary/0.log" Jul 07 00:27:52.112444 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:52.112406 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gjb9z_dc7855c6-e0ea-4385-b08f-a91bfb641524/kube-rbac-proxy/0.log" Jul 07 00:27:52.136573 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:52.136545 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gjb9z_dc7855c6-e0ea-4385-b08f-a91bfb641524/exporter/0.log" Jul 07 00:27:52.160893 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:52.160860 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gjb9z_dc7855c6-e0ea-4385-b08f-a91bfb641524/extractor/0.log" Jul 07 00:27:53.114986 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:53.114947 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" Jul 07 00:27:53.142790 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:53.142741 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s7kft/perf-node-gather-daemonset-ww55j" podStartSLOduration=8.142725672 podStartE2EDuration="8.142725672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 00:27:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 00:27:47.119121434 +0000 UTC m=+2377.381793629" watchObservedRunningTime="2026-07-07 00:27:53.142725672 +0000 UTC m=+2383.405397867" Jul 07 00:27:54.634840 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:54.634812 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-5w9s5_cc8e98a6-58cf-4d7e-a9ee-090c8939a6db/manager/0.log" Jul 07 00:27:54.813191 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:54.813105 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-hghqm_7682d489-0d6c-4b87-bdba-0cbcde4d6868/manager/0.log" Jul 07 00:27:54.884152 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:54.884125 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723040-njdwh_4a50d927-75ad-49c3-a48a-f72ff3f6027f/cleanup/0.log" Jul 07 00:27:54.911773 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:54.911742 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29723055-mmnms_54cc54a8-c744-462a-99d3-a9b43a4de909/cleanup/0.log" Jul 07 00:27:55.102521 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:55.102496 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-b456b_8414f226-13c8-429a-8a7c-c77150cc7cb1/server/0.log" Jul 07 00:27:55.195978 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:55.195929 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-tp54k_59d9a557-9bc9-4a7d-93e0-2b43f44a0d9d/manager/0.log" Jul 07 00:27:55.356457 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:55.356379 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4d578756-r2dx8_cbe40b23-970e-4072-a3a6-78985342407b/manager/0.log" Jul 07 00:27:55.376364 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:55.376336 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4d578756-r2dx8_cbe40b23-970e-4072-a3a6-78985342407b/copy-manifests/0.log" Jul 07 00:27:56.517093 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:56.517061 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t24l7_a24e7fdb-20db-4a53-83e8-caf0225145d1/istio-proxy/0.log" Jul 07 00:27:56.641126 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:56.641093 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4984cb9-8fgfb_ece0863e-d2c2-4310-b949-0bcc0e3279b0/kube-auth-proxy/0.log" Jul 07 00:27:56.665008 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:56.664978 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cbxpr_e2eb3722-5915-4084-b152-86187682a2d3/istio-proxy/0.log" Jul 07 00:27:56.702877 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:56.702844 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cbxpr_e2eb3722-5915-4084-b152-86187682a2d3/istio-proxy/1.log" Jul 07 00:27:56.949856 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:56.949783 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wrc7g_085bfed8-c216-4027-8809-981b273dd98d/openshift-lws-operator/0.log" Jul 07 00:27:58.046442 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:58.046415 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t24l7_a24e7fdb-20db-4a53-83e8-caf0225145d1/istio-proxy/0.log" Jul 07 00:27:58.181249 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:58.181168 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6c4984cb9-8fgfb_ece0863e-d2c2-4310-b949-0bcc0e3279b0/kube-auth-proxy/0.log" Jul 07 00:27:58.209596 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:58.209541 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cbxpr_e2eb3722-5915-4084-b152-86187682a2d3/istio-proxy/0.log" Jul 07 00:27:58.248036 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:27:58.248000 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-cbxpr_e2eb3722-5915-4084-b152-86187682a2d3/istio-proxy/1.log" Jul 07 00:28:03.538618 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:28:03.538590 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkdd7_341bdb2a-627b-4ea0-8236-5bf3ce3295a2/kube-multus-additional-cni-plugins/0.log" Jul 07 00:28:03.563745 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:28:03.563715 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkdd7_341bdb2a-627b-4ea0-8236-5bf3ce3295a2/egress-router-binary-copy/0.log" Jul 07 00:28:03.591089 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:28:03.591062 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkdd7_341bdb2a-627b-4ea0-8236-5bf3ce3295a2/cni-plugins/0.log" Jul 07 00:28:03.622846 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:28:03.622805 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkdd7_341bdb2a-627b-4ea0-8236-5bf3ce3295a2/bond-cni-plugin/0.log" Jul 07 00:28:03.651835 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:28:03.651806 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkdd7_341bdb2a-627b-4ea0-8236-5bf3ce3295a2/routeoverride-cni/0.log" Jul 07 00:28:03.675049 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:28:03.675025 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkdd7_341bdb2a-627b-4ea0-8236-5bf3ce3295a2/whereabouts-cni-bincopy/0.log" Jul 07 00:28:03.701528 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:28:03.701467 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zkdd7_341bdb2a-627b-4ea0-8236-5bf3ce3295a2/whereabouts-cni/0.log" Jul 07 00:28:03.778002 ip-10-0-137-188 kubenswrapper[2583]: I0707 00:28:03.777965 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ssgq9_4742585e-894a-425f-b770-15e50d8a2d82/kube-multus/0.log"