Jul 15 16:58:24.350411 ip-10-0-143-124 systemd[1]: Starting Kubernetes Kubelet... Jul 15 16:58:24.781417 ip-10-0-143-124 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:58:24.781417 ip-10-0-143-124 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 16:58:24.781417 ip-10-0-143-124 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:58:24.781417 ip-10-0-143-124 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 16:58:24.781417 ip-10-0-143-124 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:58:24.784332 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.784242 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 16:58:24.788962 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788942 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:58:24.788962 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788961 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788966 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788969 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788971 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788974 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788977 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788979 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788982 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788984 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788986 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788989 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788991 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788994 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788997 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.788999 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789002 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789004 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789006 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789015 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:58:24.789035 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789017 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789020 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789022 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789025 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789027 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789030 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789032 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789035 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789037 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789040 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789043 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789046 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789064 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789067 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789070 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789072 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789075 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789077 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789080 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:58:24.789483 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789082 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789086 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789090 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789094 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789097 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789100 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789103 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789106 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789109 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789111 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789114 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789116 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789119 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789121 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789123 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789127 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789130 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789133 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789135 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:58:24.790002 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789138 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789141 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789144 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789146 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789148 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789151 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789153 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789156 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789159 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789161 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789164 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789166 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789169 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789171 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789174 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789176 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789179 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789182 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789184 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789191 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:58:24.790579 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789193 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789197 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789200 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789203 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789205 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789208 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789211 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789214 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789217 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789220 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789223 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789226 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789228 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789231 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789234 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789237 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789239 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789241 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789244 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:58:24.791069 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789248 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789252 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789983 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789989 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789992 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789995 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.789998 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790001 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790004 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790006 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790009 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790012 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790014 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790017 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790019 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790022 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790025 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790027 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790030 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790032 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:58:24.791511 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790036 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790039 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790041 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790044 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790046 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790067 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790070 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790072 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790075 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790077 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790080 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790082 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790085 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790087 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790090 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790093 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790095 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790098 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790101 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790103 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:58:24.791981 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790106 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790110 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790114 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790116 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790119 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790122 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790125 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790128 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790130 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790133 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790136 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790138 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790142 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790148 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790150 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790153 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790155 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790158 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790161 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:58:24.792469 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790163 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790166 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790168 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790170 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790173 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790175 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790178 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790180 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790183 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790185 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790187 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790190 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790192 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790195 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790198 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790200 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790203 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790205 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790208 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790210 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:58:24.792912 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790213 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790218 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790221 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790224 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790227 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790230 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790234 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790237 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790240 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790243 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790245 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790248 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790251 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790255 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790257 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790260 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790263 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790266 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790268 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790271 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:58:24.793388 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790273 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790275 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790668 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790673 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790676 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790679 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790682 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790685 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790687 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790690 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790693 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790695 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790698 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790700 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790703 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790706 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790709 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790711 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790716 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790718 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:58:24.793893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790721 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790723 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790726 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790728 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790731 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790733 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790736 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790738 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790741 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790743 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790745 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790748 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790750 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790764 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790767 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790769 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790771 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790774 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790776 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:58:24.794398 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790779 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790782 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790784 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790787 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790789 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790791 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790794 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790796 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790799 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790801 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790803 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790808 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790810 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790812 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790816 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790820 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790823 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790825 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790828 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790831 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:58:24.794855 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790834 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790836 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790838 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790841 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790843 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790846 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790849 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790852 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790854 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790857 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790860 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790862 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790864 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790867 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790869 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790871 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790874 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790876 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790881 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790884 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:58:24.795416 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790886 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790889 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790891 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790894 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790897 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790900 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790903 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790905 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790908 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790910 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790913 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790915 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790918 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790920 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790922 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790925 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790927 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790929 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790932 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790934 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:58:24.795888 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790936 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.790938 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791017 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791025 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791031 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791035 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791040 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791043 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791047 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791066 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791071 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791074 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791078 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791082 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791085 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791088 2566 flags.go:64] FLAG: --cgroup-root="" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791091 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791094 2566 flags.go:64] FLAG: --client-ca-file="" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791097 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791099 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791103 2566 flags.go:64] FLAG: --cluster-domain="" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791106 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791109 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791112 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 16:58:24.796385 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791115 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791119 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791121 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791125 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791128 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791131 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791133 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791136 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791139 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791142 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791146 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791149 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791153 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791155 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791158 2566 flags.go:64] FLAG: --enable-server="true" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791161 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791166 2566 flags.go:64] FLAG: --event-burst="100" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791169 2566 flags.go:64] FLAG: --event-qps="50" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791173 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791176 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791179 2566 flags.go:64] FLAG: --eviction-hard="" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791183 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791186 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791189 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791192 2566 flags.go:64] FLAG: --eviction-soft="" Jul 15 16:58:24.797029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791195 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791198 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791201 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791205 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791207 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791210 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791213 2566 flags.go:64] FLAG: --feature-gates="" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791221 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791224 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791228 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791230 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791233 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791236 2566 flags.go:64] FLAG: --help="false" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791239 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-143-124.ec2.internal" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791242 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791244 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791247 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791251 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791254 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791257 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791260 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791262 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791265 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 16:58:24.797680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791269 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791272 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791276 2566 flags.go:64] FLAG: --kube-reserved="" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791279 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791282 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791285 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791287 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791290 2566 flags.go:64] FLAG: --lock-file="" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791293 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791296 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791299 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791304 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791307 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791310 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791313 2566 flags.go:64] FLAG: --logging-format="text" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791315 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791318 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791321 2566 flags.go:64] FLAG: --manifest-url="" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791324 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791328 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791331 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791335 2566 flags.go:64] FLAG: --max-pods="110" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791337 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791340 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791343 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 16:58:24.798287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791346 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791349 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791352 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791354 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791362 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791365 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791367 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791370 2566 flags.go:64] FLAG: --pod-cidr="" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791374 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791379 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791383 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791386 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791389 2566 flags.go:64] FLAG: --port="10250" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791392 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791396 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-034314c788308b44a" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791399 2566 flags.go:64] FLAG: --qos-reserved="" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791402 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791404 2566 flags.go:64] FLAG: --register-node="true" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791407 2566 flags.go:64] FLAG: --register-with-taints="" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791410 2566 flags.go:64] FLAG: --registry-burst="10" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791413 2566 flags.go:64] FLAG: --registry-qps="5" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791416 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791418 2566 flags.go:64] FLAG: --reserved-memory="" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791422 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791425 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 16:58:24.798912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791428 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791430 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791433 2566 flags.go:64] FLAG: --runonce="false" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791436 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791439 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791442 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791444 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791447 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791450 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791453 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791456 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791458 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791461 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791464 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791466 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791469 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791473 2566 flags.go:64] FLAG: --system-cgroups="" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791477 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791482 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791485 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791488 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791493 2566 flags.go:64] FLAG: --tls-min-version="" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791495 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791498 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791501 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 16:58:24.799535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791503 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791506 2566 flags.go:64] FLAG: --v="2" Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791510 2566 flags.go:64] FLAG: --version="false" Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791514 2566 flags.go:64] FLAG: --vmodule="" Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791518 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.791521 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791644 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791648 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791650 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791653 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791656 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791658 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791661 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791664 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791666 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791668 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791670 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791673 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791675 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791679 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:58:24.800124 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791681 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791684 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791687 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791691 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791696 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791699 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791702 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791704 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791708 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791710 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791713 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791715 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791718 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791720 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791723 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791725 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791727 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791730 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791732 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:58:24.800620 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791734 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791737 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791740 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791742 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791744 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791747 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791749 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791752 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791754 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791756 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791758 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791762 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791765 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791768 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791771 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791773 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791776 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791780 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791783 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:58:24.801088 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791785 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791788 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791792 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791794 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791797 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791799 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791802 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791804 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791806 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791808 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791811 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791813 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791816 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791818 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791821 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791823 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791825 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791828 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791830 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791833 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:58:24.801541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791835 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791837 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791840 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791842 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791844 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791847 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791849 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791851 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791854 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791856 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791859 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791862 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791865 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791867 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791871 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791873 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791876 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791878 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791880 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791884 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:58:24.802100 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791886 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791888 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791891 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791893 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791895 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791898 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.791900 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.792956 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.800354 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.800371 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.800391 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.800398 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800455 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800461 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:58:24.802589 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800464 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800468 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800472 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800476 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800479 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800482 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800485 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800488 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800490 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800494 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800498 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800501 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800503 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800506 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800508 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800511 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800513 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800515 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800518 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:58:24.802934 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800520 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800523 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800525 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800528 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800530 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800534 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800537 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800540 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800543 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800546 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800548 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800551 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800553 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800556 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800558 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800561 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800564 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800566 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800568 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800571 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:58:24.803452 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800573 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800575 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800578 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800580 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800583 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800585 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800588 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800590 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800592 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800595 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800598 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800600 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800603 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800605 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800607 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800610 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800612 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800615 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800617 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800619 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:58:24.804016 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800623 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800626 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800629 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800631 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800634 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800638 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800641 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800644 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800647 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800650 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800652 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800655 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800657 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800660 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800662 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800664 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800667 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800670 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800673 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:58:24.804522 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800675 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800678 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800680 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800683 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800686 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800688 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800690 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800692 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800695 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800697 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800700 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800702 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800704 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800707 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800710 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800712 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800715 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800717 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:58:24.805001 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800720 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.800725 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800841 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800845 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800848 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800851 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800854 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800856 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800858 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800861 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800863 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800865 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800868 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800871 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800873 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:58:24.805541 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800876 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800878 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800881 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800883 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800885 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800887 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800889 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800892 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800895 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800898 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800902 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800904 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800908 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800911 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800914 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800916 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800919 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800921 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800923 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:58:24.806230 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800926 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800928 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800931 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800933 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800935 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800937 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800940 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800942 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800944 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800946 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800949 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800951 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800954 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800956 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800959 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800961 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800963 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800966 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800968 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800970 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:58:24.806808 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800972 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800975 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800977 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800979 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800981 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800984 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800987 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800989 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.800992 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801006 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801010 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801013 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801016 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801018 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801022 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801026 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801029 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801031 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801034 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801037 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:58:24.807336 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801039 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801042 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801044 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801046 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801066 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801070 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801073 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801075 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801078 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801081 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801083 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801085 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801088 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801090 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801093 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801095 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801098 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801101 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801104 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:58:24.807893 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801107 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801111 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801114 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801117 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801120 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801122 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801125 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:24.801127 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.801132 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.801254 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.803613 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.805248 2566 server.go:1019] "Starting client certificate rotation" Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.805369 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:58:24.808413 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.805422 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:58:24.808784 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.808116 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:58:24.816215 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.816192 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:58:24.828196 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.828169 2566 log.go:25] "Validated CRI v1 runtime API" Jul 15 16:58:24.831244 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.831226 2566 log.go:25] "Validated CRI v1 image API" Jul 15 16:58:24.831990 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.831974 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 16:58:24.834649 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.834629 2566 fs.go:135] Filesystem UUIDs: map[18da65af-1895-4eac-88d6-1d35b80152b6:/dev/nvme0n1p3 1fab29b0-3eb5-4911-8a14-58606f9fd372:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 15 16:58:24.834703 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.834650 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 16:58:24.840086 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.840067 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:58:24.840207 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.840093 2566 manager.go:216] Machine: {Timestamp:2026-07-15 16:58:24.83859089 +0000 UTC m=+0.353846171 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3179866 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d97c4760ed67dfa1a445ff151780e SystemUUID:ec2d97c4-760e-d67d-fa1a-445ff151780e BootID:d779101d-354e-4d94-b1e1-bcf3baf2e968 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:68:00:7f:ca:b7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:68:00:7f:ca:b7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:d1:5a:f2:79:2e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 16:58:24.840207 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.840206 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 16:58:24.840311 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.840300 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 16:58:24.843765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.843726 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 16:58:24.843974 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.843768 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-124.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 16:58:24.844796 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.844783 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 16:58:24.844860 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.844803 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 16:58:24.844860 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.844822 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:58:24.845682 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.845669 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:58:24.846916 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.846902 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 16:58:24.850546 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.850528 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:58:24.850700 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.850689 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 16:58:24.850774 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.850743 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 15 16:58:24.850774 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.850758 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 16:58:24.850774 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.850773 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 16:58:24.850927 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.850787 2566 kubelet.go:398] "Adding apiserver pod source" Jul 15 16:58:24.850927 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.850802 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 16:58:24.851246 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.851232 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:58:24.851312 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.851255 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:58:24.853797 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.853781 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 16:58:24.855441 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.855426 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 16:58:24.855496 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.855456 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 16:58:24.856282 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856271 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 16:58:24.856333 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856287 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 16:58:24.856333 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856293 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 16:58:24.856333 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856298 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 16:58:24.856333 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856319 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 16:58:24.856333 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856325 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 16:58:24.856333 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856333 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 16:58:24.856494 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856339 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 16:58:24.856494 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856345 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 16:58:24.856494 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856350 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 16:58:24.856494 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856362 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 16:58:24.856494 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.856368 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 16:58:24.857598 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.857581 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 16:58:24.857598 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.857597 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 16:58:24.861350 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.861334 2566 server.go:1268] "Started kubelet" Jul 15 16:58:24.861505 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.861489 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-124.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 16:58:24.861671 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.861607 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 16:58:24.861773 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.861692 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 16:58:24.861897 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.861867 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 16:58:24.862377 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.862339 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 16:58:24.862377 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.862339 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-124.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 16:58:24.862425 ip-10-0-143-124 systemd[1]: Started Kubernetes Kubelet. Jul 15 16:58:24.863256 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.863210 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 16:58:24.869328 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.869310 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 15 16:58:24.869517 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.869498 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 16:58:24.871946 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.870920 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-124.ec2.internal.18c2850fb2b45766 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-124.ec2.internal,UID:ip-10-0-143-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-124.ec2.internal,},FirstTimestamp:2026-07-15 16:58:24.861304678 +0000 UTC m=+0.376559958,LastTimestamp:2026-07-15 16:58:24.861304678 +0000 UTC m=+0.376559958,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-124.ec2.internal,}" Jul 15 16:58:24.873347 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.873333 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 16:58:24.873403 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.873342 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 16:58:24.873449 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.873430 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:24.874136 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874120 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 16:58:24.874296 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874186 2566 factory.go:55] Registering systemd factory Jul 15 16:58:24.874396 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874386 2566 factory.go:223] Registration of the systemd container factory successfully Jul 15 16:58:24.874480 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874243 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 16:58:24.874534 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874486 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 16:58:24.874638 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874626 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 15 16:58:24.874638 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874637 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 15 16:58:24.874811 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874791 2566 factory.go:153] Registering CRI-O factory Jul 15 16:58:24.874811 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874811 2566 factory.go:223] Registration of the crio container factory successfully Jul 15 16:58:24.874945 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874863 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 16:58:24.874945 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874898 2566 factory.go:103] Registering Raw factory Jul 15 16:58:24.874945 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.874916 2566 manager.go:1196] Started watching for new ooms in manager Jul 15 16:58:24.875602 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.875586 2566 manager.go:349] Starting recovery of all containers Jul 15 16:58:24.884856 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.884686 2566 manager.go:354] Recovery completed Jul 15 16:58:24.885954 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.885926 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-124.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 16:58:24.886038 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.885945 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 16:58:24.889177 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.889161 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:58:24.891304 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.891282 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:58:24.891362 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.891318 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:58:24.891362 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.891329 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:58:24.891974 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.891963 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 16:58:24.891974 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.891973 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 16:58:24.892065 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.891988 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:58:24.892098 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.891972 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:58:24.892737 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.892724 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:58:24.892798 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.892749 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:58:24.892798 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.892760 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:58:24.893435 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.893424 2566 policy_none.go:49] "None policy: Start" Jul 15 16:58:24.893467 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.893440 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 16:58:24.893467 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.893450 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 16:58:24.894460 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.894449 2566 policy_none.go:47] "Start" Jul 15 16:58:24.901515 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.901449 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-124.ec2.internal.18c2850fb47e188b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-124.ec2.internal,UID:ip-10-0-143-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-124.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-124.ec2.internal,},FirstTimestamp:2026-07-15 16:58:24.891304075 +0000 UTC m=+0.406559355,LastTimestamp:2026-07-15 16:58:24.891304075 +0000 UTC m=+0.406559355,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-124.ec2.internal,}" Jul 15 16:58:24.904284 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.904266 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wxjlz" Jul 15 16:58:24.914431 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.913966 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-124.ec2.internal.18c2850fb47e612a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-124.ec2.internal,UID:ip-10-0-143-124.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-124.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-124.ec2.internal,},FirstTimestamp:2026-07-15 16:58:24.891322666 +0000 UTC m=+0.406577946,LastTimestamp:2026-07-15 16:58:24.891322666 +0000 UTC m=+0.406577946,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-124.ec2.internal,}" Jul 15 16:58:24.914517 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.914456 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wxjlz" Jul 15 16:58:24.932721 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.932701 2566 manager.go:337] "Starting Device Plugin manager" Jul 15 16:58:24.949710 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.932740 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 16:58:24.949710 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.932753 2566 server.go:87] "Starting device plugin registration server" Jul 15 16:58:24.949710 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.932987 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 16:58:24.949710 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.932996 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 16:58:24.949710 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.933097 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 16:58:24.949710 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.933177 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 16:58:24.949710 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.933186 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 16:58:24.949710 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.933725 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 16:58:24.949710 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.933782 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:24.978531 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.978490 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 16:58:24.979926 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.979899 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 16:58:24.980035 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.979936 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 16:58:24.980035 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.979959 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 16:58:24.980035 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:24.979994 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 16:58:24.982691 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:24.982672 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:58:25.033831 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.033757 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:58:25.034922 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.034904 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:58:25.035026 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.034938 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:58:25.035026 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.034950 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:58:25.035026 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.034973 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.056869 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.056843 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.056955 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.056880 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-124.ec2.internal\": node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:25.080106 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.080074 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal"] Jul 15 16:58:25.080250 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.080142 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:58:25.081778 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.081759 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:58:25.081881 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.081798 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:58:25.081881 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.081812 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:58:25.083374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.083357 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:58:25.083499 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.083483 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.083549 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.083519 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:58:25.084173 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.084153 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:58:25.084173 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.084176 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:58:25.084285 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.084186 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:58:25.084285 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.084197 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:58:25.084285 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.084216 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:58:25.084285 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.084230 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:58:25.085493 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.085479 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.085550 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.085505 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:58:25.086206 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.086189 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:58:25.086298 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.086212 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:58:25.086298 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.086221 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:58:25.114947 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.114922 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-124.ec2.internal\" not found" node="ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.118422 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.118404 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-124.ec2.internal\" not found" node="ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.122274 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.122258 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:25.176651 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.176620 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0870127af64eb5679c3501716c462db4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal\" (UID: \"0870127af64eb5679c3501716c462db4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.176651 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.176653 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0870127af64eb5679c3501716c462db4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal\" (UID: \"0870127af64eb5679c3501716c462db4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.176781 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.176671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0fb8ca27f1fff44d36146fb73f1ead9-config\") pod \"kube-apiserver-proxy-ip-10-0-143-124.ec2.internal\" (UID: \"f0fb8ca27f1fff44d36146fb73f1ead9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.222411 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.222380 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:25.277545 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.277510 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0870127af64eb5679c3501716c462db4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal\" (UID: \"0870127af64eb5679c3501716c462db4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.277665 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.277547 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0870127af64eb5679c3501716c462db4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal\" (UID: \"0870127af64eb5679c3501716c462db4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.277665 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.277612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0fb8ca27f1fff44d36146fb73f1ead9-config\") pod \"kube-apiserver-proxy-ip-10-0-143-124.ec2.internal\" (UID: \"f0fb8ca27f1fff44d36146fb73f1ead9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.277665 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.277621 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0870127af64eb5679c3501716c462db4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal\" (UID: \"0870127af64eb5679c3501716c462db4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.277762 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.277678 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f0fb8ca27f1fff44d36146fb73f1ead9-config\") pod \"kube-apiserver-proxy-ip-10-0-143-124.ec2.internal\" (UID: \"f0fb8ca27f1fff44d36146fb73f1ead9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.277762 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.277709 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0870127af64eb5679c3501716c462db4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal\" (UID: \"0870127af64eb5679c3501716c462db4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.322878 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.322815 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:25.418571 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.418538 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.421358 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.421340 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.423805 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.423790 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:25.523900 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.523856 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:25.624483 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.624454 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:25.725101 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.725064 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:25.805686 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.805650 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 16:58:25.806272 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.805840 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 16:58:25.825945 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:25.825911 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-124.ec2.internal\" not found" Jul 15 16:58:25.875721 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.875639 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:58:25.877341 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.877326 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:58:25.918962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.918917 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 16:53:26 +0000 UTC" deadline="2027-12-11 20:52:31.47679355 +0000 UTC" Jul 15 16:58:25.919133 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.918961 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12339h54m5.557836184s" Jul 15 16:58:25.973673 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.973609 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" Jul 15 16:58:25.990174 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.990147 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:58:25.991297 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:25.991278 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal" Jul 15 16:58:26.001551 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:26.001512 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:58:26.012943 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:26.012911 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0fb8ca27f1fff44d36146fb73f1ead9.slice/crio-43501b9ad53f7b2c093a9fc45f37a9c1af97c7ffe495dedba0dbf9615e3ca5c3 WatchSource:0}: Error finding container 43501b9ad53f7b2c093a9fc45f37a9c1af97c7ffe495dedba0dbf9615e3ca5c3: Status 404 returned error can't find the container with id 43501b9ad53f7b2c093a9fc45f37a9c1af97c7ffe495dedba0dbf9615e3ca5c3 Jul 15 16:58:26.013192 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:26.013171 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0870127af64eb5679c3501716c462db4.slice/crio-d55e7d7af7f718f799fef5bd4e662cddd4b2176727505674be26db75e5244205 WatchSource:0}: Error finding container d55e7d7af7f718f799fef5bd4e662cddd4b2176727505674be26db75e5244205: Status 404 returned error can't find the container with id d55e7d7af7f718f799fef5bd4e662cddd4b2176727505674be26db75e5244205 Jul 15 16:58:26.018386 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:26.018368 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 16:58:27.576263 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:27.576231 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:58:28.053563 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.053531 2566 apiserver.go:52] "Watching apiserver" Jul 15 16:58:28.065835 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.065804 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 16:58:28.066781 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.066539 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-wb5hg","kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd","openshift-cluster-node-tuning-operator/tuned-598cx","openshift-dns/node-resolver-t8xnk","openshift-multus/network-metrics-daemon-w7n7w","openshift-network-diagnostics/network-check-target-m4wq5","kube-system/global-pull-secret-syncer-d48rf","kube-system/konnectivity-agent-w9zmt","openshift-image-registry/node-ca-2pxcv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal","openshift-multus/multus-additional-cni-plugins-mxzc4","openshift-multus/multus-lbkks","openshift-network-operator/iptables-alerter-4rm7b"] Jul 15 16:58:28.068741 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.068721 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:28.068853 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.068815 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:28.070070 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.070052 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.071235 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.071214 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.072507 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.072487 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.073181 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.073162 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 16:58:28.073789 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.073770 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.074993 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.074969 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 16:58:28.075315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.075294 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fcnm4\"" Jul 15 16:58:28.075446 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.075434 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 16:58:28.076064 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.076042 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 16:58:28.077166 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.077144 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 16:58:28.077347 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.077330 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 16:58:28.077620 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.077588 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:28.077695 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.077660 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:58:28.077695 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.077665 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:28.077828 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.077813 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 16:58:28.077828 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.077825 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 16:58:28.078059 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.078043 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 16:58:28.078222 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.078205 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 16:58:28.078564 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.078549 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:28.080010 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.079988 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.080254 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.080233 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6njq9\"" Jul 15 16:58:28.080474 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.080455 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 16:58:28.080828 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.080813 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-92tpp\"" Jul 15 16:58:28.081135 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.081119 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-42kjl\"" Jul 15 16:58:28.081314 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.081296 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.081549 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.081532 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 16:58:28.081958 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.081942 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 16:58:28.082314 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.082297 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.083870 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.083840 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.083957 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.083904 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:28.084999 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.084981 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.087233 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087211 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t9hrp\"" Jul 15 16:58:28.087673 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087652 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-var-lib-kubelet\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.087784 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087685 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-run-k8s-cni-cncf-io\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.087784 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087712 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-hostroot\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.087784 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087752 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2d7dc39f-d2fc-42e9-9338-decd2e825747-hosts-file\") pod \"node-resolver-t8xnk\" (UID: \"2d7dc39f-d2fc-42e9-9338-decd2e825747\") " pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.087784 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-tuned\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.087972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087792 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:28.087972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087809 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-etc-selinux\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.087972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087839 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-modprobe-d\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.087972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087868 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-kubernetes\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.087972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087886 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-sysctl-conf\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.087972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087901 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-lib-modules\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.087972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087917 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-host\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.087972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087939 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwjrl\" (UniqueName: \"kubernetes.io/projected/c7342b5e-c845-43f4-9298-fd94f90fdc8a-kube-api-access-fwjrl\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.087972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087963 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-socket-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.087989 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-cni-binary-copy\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088036 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-var-lib-cni-multus\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088065 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-run-multus-certs\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088086 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-etc-kubernetes\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088107 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-registration-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088123 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-var-lib-kubelet\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088137 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-device-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088164 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-systemd\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-run\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/56812b13-8b82-41aa-83a9-0f5cf373ddfa-konnectivity-ca\") pod \"konnectivity-agent-w9zmt\" (UID: \"56812b13-8b82-41aa-83a9-0f5cf373ddfa\") " pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088247 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-socket-dir-parent\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088271 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088323 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd9kb\" (UniqueName: \"kubernetes.io/projected/f442938c-8777-4cb5-ab3e-1f5dfaef9248-kube-api-access-sd9kb\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088350 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-cni-dir\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.088374 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-daemon-config\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s4xn\" (UniqueName: \"kubernetes.io/projected/2d7dc39f-d2fc-42e9-9338-decd2e825747-kube-api-access-6s4xn\") pod \"node-resolver-t8xnk\" (UID: \"2d7dc39f-d2fc-42e9-9338-decd2e825747\") " pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088423 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5csv\" (UniqueName: \"kubernetes.io/projected/1d2712c7-d910-4a49-900d-ae83e771b3c1-kube-api-access-w5csv\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088439 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088455 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-system-cni-dir\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088474 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-cnibin\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088487 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-run-netns\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088500 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcwmp\" (UniqueName: \"kubernetes.io/projected/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-kube-api-access-bcwmp\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088513 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-sysctl-d\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088534 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-sys\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088556 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7342b5e-c845-43f4-9298-fd94f90fdc8a-tmp\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/56812b13-8b82-41aa-83a9-0f5cf373ddfa-agent-certs\") pod \"konnectivity-agent-w9zmt\" (UID: \"56812b13-8b82-41aa-83a9-0f5cf373ddfa\") " pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088586 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-os-release\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-var-lib-cni-bin\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088654 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-conf-dir\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d7dc39f-d2fc-42e9-9338-decd2e825747-tmp-dir\") pod \"node-resolver-t8xnk\" (UID: \"2d7dc39f-d2fc-42e9-9338-decd2e825747\") " pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088709 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-sys-fs\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.089074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.088730 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-sysconfig\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.091651 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.091628 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:58:28.091651 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.091647 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 16:58:28.091812 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.091699 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 16:58:28.091812 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.091774 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 16:58:28.091910 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.091821 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 16:58:28.091910 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.091841 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-84ncz\"" Jul 15 16:58:28.091910 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.091846 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kdwbd\"" Jul 15 16:58:28.091910 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.091904 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 16:58:28.092129 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092111 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 16:58:28.092217 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092132 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wtgrq\"" Jul 15 16:58:28.092217 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092111 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 16:58:28.092217 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092197 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 16:58:28.092698 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092678 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pjc4n\"" Jul 15 16:58:28.092781 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092752 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 16:58:28.092850 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092688 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 16:58:28.092850 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092830 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 16:58:28.092947 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092784 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 16:58:28.092994 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.092968 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 16:58:28.100088 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.100061 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:58:28.164355 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.164316 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2nz8w" Jul 15 16:58:28.170939 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.170903 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2nz8w" Jul 15 16:58:28.177077 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.177048 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 16:58:28.187029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.186971 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal" event={"ID":"f0fb8ca27f1fff44d36146fb73f1ead9","Type":"ContainerStarted","Data":"43501b9ad53f7b2c093a9fc45f37a9c1af97c7ffe495dedba0dbf9615e3ca5c3"} Jul 15 16:58:28.188929 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.188904 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:28.189051 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.188934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-kubernetes\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.189051 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.188953 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-var-lib-cni-multus\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.189051 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.188970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-run-multus-certs\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.189051 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.188986 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-registration-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.189051 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-ovnkube-script-lib\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.189051 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189006 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" event={"ID":"0870127af64eb5679c3501716c462db4","Type":"ContainerStarted","Data":"d55e7d7af7f718f799fef5bd4e662cddd4b2176727505674be26db75e5244205"} Jul 15 16:58:28.189051 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189018 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-run\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.189051 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189032 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-kubelet\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189071 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd5089e4-2941-42c5-9203-d6d3d7319ffc-host-slash\") pod \"iptables-alerter-4rm7b\" (UID: \"dd5089e4-2941-42c5-9203-d6d3d7319ffc\") " pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189085 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-cnibin\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189100 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-socket-dir-parent\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189116 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd9kb\" (UniqueName: \"kubernetes.io/projected/f442938c-8777-4cb5-ab3e-1f5dfaef9248-kube-api-access-sd9kb\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189113 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-run-multus-certs\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189138 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-slash\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189157 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-system-cni-dir\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189175 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-cni-dir\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189190 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-daemon-config\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drxlj\" (UniqueName: \"kubernetes.io/projected/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-kube-api-access-drxlj\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d65117a-a7e7-4feb-9a0f-f05010d0c737-cni-binary-copy\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.189230 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189235 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d65117a-a7e7-4feb-9a0f-f05010d0c737-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-run-systemd\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189266 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-system-cni-dir\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.189409 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.189297 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs podName:1d2712c7-d910-4a49-900d-ae83e771b3c1 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:28.68927315 +0000 UTC m=+3.002978022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs") pod "network-metrics-daemon-w7n7w" (UID: "1d2712c7-d910-4a49-900d-ae83e771b3c1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189320 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-cnibin\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189336 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-system-cni-dir\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-run-netns\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7342b5e-c845-43f4-9298-fd94f90fdc8a-tmp\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189404 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-node-log\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189416 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-run\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-run-ovn-kubernetes\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189454 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd5089e4-2941-42c5-9203-d6d3d7319ffc-iptables-alerter-script\") pod \"iptables-alerter-4rm7b\" (UID: \"dd5089e4-2941-42c5-9203-d6d3d7319ffc\") " pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189527 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-kubernetes\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189566 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-var-lib-cni-multus\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-os-release\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-conf-dir\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d7dc39f-d2fc-42e9-9338-decd2e825747-tmp-dir\") pod \"node-resolver-t8xnk\" (UID: \"2d7dc39f-d2fc-42e9-9338-decd2e825747\") " pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189379 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-registration-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-sysconfig\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189709 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-cni-dir\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189737 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-sysconfig\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.190152 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189741 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-cnibin\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189752 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-socket-dir-parent\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189775 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-conf-dir\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189801 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-os-release\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189940 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-ovn-node-metrics-cert\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.189971 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-os-release\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190002 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-run-netns\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190002 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d65117a-a7e7-4feb-9a0f-f05010d0c737-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190035 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190041 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-var-lib-kubelet\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190071 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-hostroot\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190092 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2d7dc39f-d2fc-42e9-9338-decd2e825747-hosts-file\") pod \"node-resolver-t8xnk\" (UID: \"2d7dc39f-d2fc-42e9-9338-decd2e825747\") " pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190115 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-tuned\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190124 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-var-lib-kubelet\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190138 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-etc-selinux\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190176 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-hostroot\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190181 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d7dc39f-d2fc-42e9-9338-decd2e825747-tmp-dir\") pod \"node-resolver-t8xnk\" (UID: \"2d7dc39f-d2fc-42e9-9338-decd2e825747\") " pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190184 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-modprobe-d\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.190962 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190236 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-sysctl-conf\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190265 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-lib-modules\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190274 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-etc-selinux\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2d7dc39f-d2fc-42e9-9338-decd2e825747-hosts-file\") pod \"node-resolver-t8xnk\" (UID: \"2d7dc39f-d2fc-42e9-9338-decd2e825747\") " pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-host\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190342 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwjrl\" (UniqueName: \"kubernetes.io/projected/c7342b5e-c845-43f4-9298-fd94f90fdc8a-kube-api-access-fwjrl\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190350 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-modprobe-d\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190370 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-etc-openvswitch\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190375 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-sysctl-conf\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190415 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-socket-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190420 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-host\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190436 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-cni-binary-copy\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190454 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-lib-modules\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-etc-kubernetes\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-var-lib-kubelet\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190666 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-log-socket\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prsj4\" (UniqueName: \"kubernetes.io/projected/dd5089e4-2941-42c5-9203-d6d3d7319ffc-kube-api-access-prsj4\") pod \"iptables-alerter-4rm7b\" (UID: \"dd5089e4-2941-42c5-9203-d6d3d7319ffc\") " pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.191765 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190684 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-socket-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190708 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25eeab2a-5a16-4d82-a085-22f22a60c078-host\") pod \"node-ca-2pxcv\" (UID: \"25eeab2a-5a16-4d82-a085-22f22a60c078\") " pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190756 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-var-lib-kubelet\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190784 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-etc-kubernetes\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190811 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-device-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190917 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-cni-binary-copy\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-systemd\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.190980 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-device-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191019 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-systemd\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191039 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-multus-daemon-config\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/56812b13-8b82-41aa-83a9-0f5cf373ddfa-konnectivity-ca\") pod \"konnectivity-agent-w9zmt\" (UID: \"56812b13-8b82-41aa-83a9-0f5cf373ddfa\") " pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191210 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-cni-bin\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7c43931a-91df-49ea-bea3-10e907fe1b5c-dbus\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191258 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191283 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-var-lib-openvswitch\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191309 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t89t8\" (UniqueName: \"kubernetes.io/projected/5d65117a-a7e7-4feb-9a0f-f05010d0c737-kube-api-access-t89t8\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.192514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191332 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s4xn\" (UniqueName: \"kubernetes.io/projected/2d7dc39f-d2fc-42e9-9338-decd2e825747-kube-api-access-6s4xn\") pod \"node-resolver-t8xnk\" (UID: \"2d7dc39f-d2fc-42e9-9338-decd2e825747\") " pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191343 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5csv\" (UniqueName: \"kubernetes.io/projected/1d2712c7-d910-4a49-900d-ae83e771b3c1-kube-api-access-w5csv\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191399 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-ovnkube-config\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191423 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-env-overrides\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191444 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7c43931a-91df-49ea-bea3-10e907fe1b5c-kubelet-config\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191465 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191486 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q67bx\" (UniqueName: \"kubernetes.io/projected/25eeab2a-5a16-4d82-a085-22f22a60c078-kube-api-access-q67bx\") pod \"node-ca-2pxcv\" (UID: \"25eeab2a-5a16-4d82-a085-22f22a60c078\") " pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191516 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcwmp\" (UniqueName: \"kubernetes.io/projected/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-kube-api-access-bcwmp\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191540 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-sysctl-d\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191564 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-sys\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191588 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/56812b13-8b82-41aa-83a9-0f5cf373ddfa-agent-certs\") pod \"konnectivity-agent-w9zmt\" (UID: \"56812b13-8b82-41aa-83a9-0f5cf373ddfa\") " pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191643 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-cni-netd\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191671 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-var-lib-cni-bin\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-sys-fs\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191707 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/56812b13-8b82-41aa-83a9-0f5cf373ddfa-konnectivity-ca\") pod \"konnectivity-agent-w9zmt\" (UID: \"56812b13-8b82-41aa-83a9-0f5cf373ddfa\") " pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:28.193306 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191723 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-run-openvswitch\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191738 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-sysctl-d\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191768 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-run-ovn\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191796 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f442938c-8777-4cb5-ab3e-1f5dfaef9248-sys-fs\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191797 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191836 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/25eeab2a-5a16-4d82-a085-22f22a60c078-serviceca\") pod \"node-ca-2pxcv\" (UID: \"25eeab2a-5a16-4d82-a085-22f22a60c078\") " pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191851 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c7342b5e-c845-43f4-9298-fd94f90fdc8a-sys\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-run-k8s-cni-cncf-io\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191888 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-systemd-units\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191907 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-run-netns\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191940 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-run-k8s-cni-cncf-io\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.191994 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-host-var-lib-cni-bin\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.193815 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c7342b5e-c845-43f4-9298-fd94f90fdc8a-etc-tuned\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.194003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.193870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7342b5e-c845-43f4-9298-fd94f90fdc8a-tmp\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.195059 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.195037 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/56812b13-8b82-41aa-83a9-0f5cf373ddfa-agent-certs\") pod \"konnectivity-agent-w9zmt\" (UID: \"56812b13-8b82-41aa-83a9-0f5cf373ddfa\") " pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:28.208436 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.208411 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd9kb\" (UniqueName: \"kubernetes.io/projected/f442938c-8777-4cb5-ab3e-1f5dfaef9248-kube-api-access-sd9kb\") pod \"aws-ebs-csi-driver-node-5f2xd\" (UID: \"f442938c-8777-4cb5-ab3e-1f5dfaef9248\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.221487 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.221454 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:58:28.221487 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.221488 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:58:28.221906 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.221511 2566 projected.go:196] Error preparing data for projected volume kube-api-access-zbnxw for pod openshift-network-diagnostics/network-check-target-m4wq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:28.221906 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.221789 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw podName:fc473a95-bf8d-46ac-b047-8be9d1c1efc3 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:28.721767751 +0000 UTC m=+3.035472622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zbnxw" (UniqueName: "kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw") pod "network-check-target-m4wq5" (UID: "fc473a95-bf8d-46ac-b047-8be9d1c1efc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:28.223856 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.223825 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5csv\" (UniqueName: \"kubernetes.io/projected/1d2712c7-d910-4a49-900d-ae83e771b3c1-kube-api-access-w5csv\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:28.225278 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.225244 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s4xn\" (UniqueName: \"kubernetes.io/projected/2d7dc39f-d2fc-42e9-9338-decd2e825747-kube-api-access-6s4xn\") pod \"node-resolver-t8xnk\" (UID: \"2d7dc39f-d2fc-42e9-9338-decd2e825747\") " pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.231012 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.230978 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwjrl\" (UniqueName: \"kubernetes.io/projected/c7342b5e-c845-43f4-9298-fd94f90fdc8a-kube-api-access-fwjrl\") pod \"tuned-598cx\" (UID: \"c7342b5e-c845-43f4-9298-fd94f90fdc8a\") " pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.231161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.231133 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcwmp\" (UniqueName: \"kubernetes.io/projected/aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7-kube-api-access-bcwmp\") pod \"multus-lbkks\" (UID: \"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7\") " pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.292876 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.292839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-etc-openvswitch\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293063 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.292884 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-log-socket\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293063 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.292910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prsj4\" (UniqueName: \"kubernetes.io/projected/dd5089e4-2941-42c5-9203-d6d3d7319ffc-kube-api-access-prsj4\") pod \"iptables-alerter-4rm7b\" (UID: \"dd5089e4-2941-42c5-9203-d6d3d7319ffc\") " pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.293063 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.292935 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25eeab2a-5a16-4d82-a085-22f22a60c078-host\") pod \"node-ca-2pxcv\" (UID: \"25eeab2a-5a16-4d82-a085-22f22a60c078\") " pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.293063 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.292958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.293063 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.292961 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-log-socket\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293063 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.292986 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-cni-bin\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293063 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7c43931a-91df-49ea-bea3-10e907fe1b5c-dbus\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.293063 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293017 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/25eeab2a-5a16-4d82-a085-22f22a60c078-host\") pod \"node-ca-2pxcv\" (UID: \"25eeab2a-5a16-4d82-a085-22f22a60c078\") " pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.293063 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.292959 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-etc-openvswitch\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293076 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-var-lib-openvswitch\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293033 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-var-lib-openvswitch\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293096 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-cni-bin\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293116 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t89t8\" (UniqueName: \"kubernetes.io/projected/5d65117a-a7e7-4feb-9a0f-f05010d0c737-kube-api-access-t89t8\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293152 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-ovnkube-config\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293152 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-env-overrides\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293207 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7c43931a-91df-49ea-bea3-10e907fe1b5c-kubelet-config\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293225 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293239 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q67bx\" (UniqueName: \"kubernetes.io/projected/25eeab2a-5a16-4d82-a085-22f22a60c078-kube-api-access-q67bx\") pod \"node-ca-2pxcv\" (UID: \"25eeab2a-5a16-4d82-a085-22f22a60c078\") " pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293252 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7c43931a-91df-49ea-bea3-10e907fe1b5c-dbus\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293259 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-cni-netd\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293286 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-run-openvswitch\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293301 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7c43931a-91df-49ea-bea3-10e907fe1b5c-kubelet-config\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.293472 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.293397 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.293492 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret podName:7c43931a-91df-49ea-bea3-10e907fe1b5c nodeName:}" failed. No retries permitted until 2026-07-15 16:58:28.793444038 +0000 UTC m=+3.107148924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret") pod "global-pull-secret-syncer-d48rf" (UID: "7c43931a-91df-49ea-bea3-10e907fe1b5c") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293507 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-cni-netd\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293550 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-run-openvswitch\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293572 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-run-ovn\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293304 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-run-ovn\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293616 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293641 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/25eeab2a-5a16-4d82-a085-22f22a60c078-serviceca\") pod \"node-ca-2pxcv\" (UID: \"25eeab2a-5a16-4d82-a085-22f22a60c078\") " pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-systemd-units\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293676 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-run-netns\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293703 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-ovnkube-script-lib\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293698 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293726 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-kubelet\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd5089e4-2941-42c5-9203-d6d3d7319ffc-host-slash\") pod \"iptables-alerter-4rm7b\" (UID: \"dd5089e4-2941-42c5-9203-d6d3d7319ffc\") " pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293750 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-systemd-units\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293752 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-env-overrides\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294161 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293759 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-cnibin\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293809 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-slash\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293821 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd5089e4-2941-42c5-9203-d6d3d7319ffc-host-slash\") pod \"iptables-alerter-4rm7b\" (UID: \"dd5089e4-2941-42c5-9203-d6d3d7319ffc\") " pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293834 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-system-cni-dir\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293856 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-slash\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293863 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drxlj\" (UniqueName: \"kubernetes.io/projected/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-kube-api-access-drxlj\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d65117a-a7e7-4feb-9a0f-f05010d0c737-cni-binary-copy\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293910 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-system-cni-dir\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d65117a-a7e7-4feb-9a0f-f05010d0c737-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293936 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-run-systemd\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293963 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-node-log\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293986 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-run-ovn-kubernetes\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd5089e4-2941-42c5-9203-d6d3d7319ffc-iptables-alerter-script\") pod \"iptables-alerter-4rm7b\" (UID: \"dd5089e4-2941-42c5-9203-d6d3d7319ffc\") " pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-ovn-node-metrics-cert\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-os-release\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294081 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-run-systemd\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d65117a-a7e7-4feb-9a0f-f05010d0c737-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.294877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294232 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/25eeab2a-5a16-4d82-a085-22f22a60c078-serviceca\") pod \"node-ca-2pxcv\" (UID: \"25eeab2a-5a16-4d82-a085-22f22a60c078\") " pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294233 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-ovnkube-script-lib\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294270 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-kubelet\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294311 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-run-ovn-kubernetes\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293776 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-ovnkube-config\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294362 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-node-log\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294532 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5d65117a-a7e7-4feb-9a0f-f05010d0c737-cni-binary-copy\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293781 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-cnibin\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294597 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5d65117a-a7e7-4feb-9a0f-f05010d0c737-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.293889 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-host-run-netns\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294624 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5d65117a-a7e7-4feb-9a0f-f05010d0c737-os-release\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.294790 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5d65117a-a7e7-4feb-9a0f-f05010d0c737-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.295532 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.295067 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dd5089e4-2941-42c5-9203-d6d3d7319ffc-iptables-alerter-script\") pod \"iptables-alerter-4rm7b\" (UID: \"dd5089e4-2941-42c5-9203-d6d3d7319ffc\") " pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.296809 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.296790 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-ovn-node-metrics-cert\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.310940 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.310861 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:58:28.316912 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.316881 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t89t8\" (UniqueName: \"kubernetes.io/projected/5d65117a-a7e7-4feb-9a0f-f05010d0c737-kube-api-access-t89t8\") pod \"multus-additional-cni-plugins-mxzc4\" (UID: \"5d65117a-a7e7-4feb-9a0f-f05010d0c737\") " pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.317207 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.317189 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prsj4\" (UniqueName: \"kubernetes.io/projected/dd5089e4-2941-42c5-9203-d6d3d7319ffc-kube-api-access-prsj4\") pod \"iptables-alerter-4rm7b\" (UID: \"dd5089e4-2941-42c5-9203-d6d3d7319ffc\") " pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.319655 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.319627 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drxlj\" (UniqueName: \"kubernetes.io/projected/027a10e2-2eaa-42fa-8d0c-ceb61f74a71d-kube-api-access-drxlj\") pod \"ovnkube-node-wb5hg\" (UID: \"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d\") " pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.320693 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.320674 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q67bx\" (UniqueName: \"kubernetes.io/projected/25eeab2a-5a16-4d82-a085-22f22a60c078-kube-api-access-q67bx\") pod \"node-ca-2pxcv\" (UID: \"25eeab2a-5a16-4d82-a085-22f22a60c078\") " pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.385623 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.385576 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lbkks" Jul 15 16:58:28.394045 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.394019 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" Jul 15 16:58:28.401904 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.401876 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-598cx" Jul 15 16:58:28.408944 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.408918 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t8xnk" Jul 15 16:58:28.415446 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.415418 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:28.423660 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.423636 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:28.432420 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.432393 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4rm7b" Jul 15 16:58:28.441366 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.441341 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2pxcv" Jul 15 16:58:28.447460 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.447436 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" Jul 15 16:58:28.697987 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.697952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:28.698162 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.698124 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:28.698218 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.698206 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs podName:1d2712c7-d910-4a49-900d-ae83e771b3c1 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:29.698182165 +0000 UTC m=+4.011887048 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs") pod "network-metrics-daemon-w7n7w" (UID: "1d2712c7-d910-4a49-900d-ae83e771b3c1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:28.798745 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.798701 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:28.798932 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:28.798753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:28.798932 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.798853 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:28.798932 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.798883 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:58:28.798932 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.798907 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:58:28.798932 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.798920 2566 projected.go:196] Error preparing data for projected volume kube-api-access-zbnxw for pod openshift-network-diagnostics/network-check-target-m4wq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:28.799145 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.798907 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret podName:7c43931a-91df-49ea-bea3-10e907fe1b5c nodeName:}" failed. No retries permitted until 2026-07-15 16:58:29.798892812 +0000 UTC m=+4.112597684 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret") pod "global-pull-secret-syncer-d48rf" (UID: "7c43931a-91df-49ea-bea3-10e907fe1b5c") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:28.799145 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:28.799053 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw podName:fc473a95-bf8d-46ac-b047-8be9d1c1efc3 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:29.799036825 +0000 UTC m=+4.112741701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbnxw" (UniqueName: "kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw") pod "network-check-target-m4wq5" (UID: "fc473a95-bf8d-46ac-b047-8be9d1c1efc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:29.174562 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:29.174531 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:53:28 +0000 UTC" deadline="2027-12-23 01:34:08.829155907 +0000 UTC" Jul 15 16:58:29.174887 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:29.174564 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12608h35m39.654595269s" Jul 15 16:58:29.198980 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:29.198954 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7342b5e_c845_43f4_9298_fd94f90fdc8a.slice/crio-6d4abfca6fa5b17f598944ce29cb378f6ae9eb6208cadb2ceb28402b03fa9053 WatchSource:0}: Error finding container 6d4abfca6fa5b17f598944ce29cb378f6ae9eb6208cadb2ceb28402b03fa9053: Status 404 returned error can't find the container with id 6d4abfca6fa5b17f598944ce29cb378f6ae9eb6208cadb2ceb28402b03fa9053 Jul 15 16:58:29.200621 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:58:29.200447 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5089e4_2941_42c5_9203_d6d3d7319ffc.slice/crio-1cc52ae47906aba3b28be1eb09867a7fe6ea1fd2146ee2a9be7e46a84e706737 WatchSource:0}: Error finding container 1cc52ae47906aba3b28be1eb09867a7fe6ea1fd2146ee2a9be7e46a84e706737: Status 404 returned error can't find the container with id 1cc52ae47906aba3b28be1eb09867a7fe6ea1fd2146ee2a9be7e46a84e706737 Jul 15 16:58:29.706254 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:29.706171 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:29.706411 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:29.706359 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:29.706475 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:29.706423 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs podName:1d2712c7-d910-4a49-900d-ae83e771b3c1 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:31.706404433 +0000 UTC m=+6.020109308 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs") pod "network-metrics-daemon-w7n7w" (UID: "1d2712c7-d910-4a49-900d-ae83e771b3c1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:29.807400 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:29.807360 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:29.807575 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:29.807416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:29.807575 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:29.807571 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:29.807782 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:29.807767 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:58:29.807859 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:29.807788 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:58:29.807859 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:29.807801 2566 projected.go:196] Error preparing data for projected volume kube-api-access-zbnxw for pod openshift-network-diagnostics/network-check-target-m4wq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:29.807859 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:29.807853 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret podName:7c43931a-91df-49ea-bea3-10e907fe1b5c nodeName:}" failed. No retries permitted until 2026-07-15 16:58:31.807836086 +0000 UTC m=+6.121540958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret") pod "global-pull-secret-syncer-d48rf" (UID: "7c43931a-91df-49ea-bea3-10e907fe1b5c") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:29.808015 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:29.807871 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw podName:fc473a95-bf8d-46ac-b047-8be9d1c1efc3 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:31.807862409 +0000 UTC m=+6.121567283 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbnxw" (UniqueName: "kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw") pod "network-check-target-m4wq5" (UID: "fc473a95-bf8d-46ac-b047-8be9d1c1efc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:30.175597 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.175498 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:53:28 +0000 UTC" deadline="2028-01-08 04:39:04.892235638 +0000 UTC" Jul 15 16:58:30.175597 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.175572 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12995h40m34.716667913s" Jul 15 16:58:30.183470 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.182741 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:30.183470 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:30.182881 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:30.183470 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.183325 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:30.183470 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:30.183419 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:30.183984 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.183953 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:30.184102 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:30.184046 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:30.204578 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.204534 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" event={"ID":"f442938c-8777-4cb5-ab3e-1f5dfaef9248","Type":"ContainerStarted","Data":"f5a4dd563536f1fd889a5a40dd0ab1669ffcb8dcd37ec9cd408a0229a63d8887"} Jul 15 16:58:30.220135 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.220097 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t8xnk" event={"ID":"2d7dc39f-d2fc-42e9-9338-decd2e825747","Type":"ContainerStarted","Data":"23f3c9a550230cb72813c7af67bc72e223e14cdac2393583855a8218f54dc810"} Jul 15 16:58:30.232325 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.232273 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4rm7b" event={"ID":"dd5089e4-2941-42c5-9203-d6d3d7319ffc","Type":"ContainerStarted","Data":"1cc52ae47906aba3b28be1eb09867a7fe6ea1fd2146ee2a9be7e46a84e706737"} Jul 15 16:58:30.247157 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.247041 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pxcv" event={"ID":"25eeab2a-5a16-4d82-a085-22f22a60c078","Type":"ContainerStarted","Data":"bd02088ecabb4fde2b842d93077b796aaa63aa9fe0ffbfaf7722816269f0c82d"} Jul 15 16:58:30.279469 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.278763 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal" event={"ID":"f0fb8ca27f1fff44d36146fb73f1ead9","Type":"ContainerStarted","Data":"ff172d969bbffd84adb0ea436d2373d4e61a0f0e74a5f6c9a952f40d5be1b214"} Jul 15 16:58:30.299505 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.299300 2566 generic.go:356] "Generic (PLEG): container finished" podID="0870127af64eb5679c3501716c462db4" containerID="506741ab6c651271ef6144b5c5254d3a77e73fae47edcc5a87cc5c6cb31c97c1" exitCode=0 Jul 15 16:58:30.299505 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.299401 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" event={"ID":"0870127af64eb5679c3501716c462db4","Type":"ContainerDied","Data":"506741ab6c651271ef6144b5c5254d3a77e73fae47edcc5a87cc5c6cb31c97c1"} Jul 15 16:58:30.300755 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.300701 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-124.ec2.internal" podStartSLOduration=3.30068175 podStartE2EDuration="3.30068175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:30.300559447 +0000 UTC m=+4.614264330" watchObservedRunningTime="2026-07-15 16:58:30.30068175 +0000 UTC m=+4.614386642" Jul 15 16:58:30.309989 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.309944 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" event={"ID":"5d65117a-a7e7-4feb-9a0f-f05010d0c737","Type":"ContainerStarted","Data":"09baa6f2d34116864f37446f86a4e41aa9615c49ec79eaef2f7c6ffb1d3b6822"} Jul 15 16:58:30.324812 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.324731 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w9zmt" event={"ID":"56812b13-8b82-41aa-83a9-0f5cf373ddfa","Type":"ContainerStarted","Data":"fe77567ac098d82564f123d098158f5aa87f9bbf9ddae9fe2906ee63ef2efceb"} Jul 15 16:58:30.335378 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.334930 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerStarted","Data":"3a361cd1c748e1656c12a995e7bbe00f242b459ed0ac23809eaa71b33b4fd1e1"} Jul 15 16:58:30.349719 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.349680 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lbkks" event={"ID":"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7","Type":"ContainerStarted","Data":"39eb4ae0f56878aa7e6237946c0d9e434bc88183154e013f55e0fca6c00e7edb"} Jul 15 16:58:30.355522 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:30.355434 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-598cx" event={"ID":"c7342b5e-c845-43f4-9298-fd94f90fdc8a","Type":"ContainerStarted","Data":"6d4abfca6fa5b17f598944ce29cb378f6ae9eb6208cadb2ceb28402b03fa9053"} Jul 15 16:58:31.370382 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:31.370345 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" event={"ID":"0870127af64eb5679c3501716c462db4","Type":"ContainerStarted","Data":"f46de3411cc89eca4d835e9cd0e3e662a3b239ce1ddb76451667b7ad095a8e7b"} Jul 15 16:58:31.387398 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:31.387339 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-124.ec2.internal" podStartSLOduration=4.387318104 podStartE2EDuration="4.387318104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:31.38708891 +0000 UTC m=+5.700793799" watchObservedRunningTime="2026-07-15 16:58:31.387318104 +0000 UTC m=+5.701022997" Jul 15 16:58:31.729249 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:31.729140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:31.729450 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:31.729338 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:31.729450 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:31.729411 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs podName:1d2712c7-d910-4a49-900d-ae83e771b3c1 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:35.729381019 +0000 UTC m=+10.043085896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs") pod "network-metrics-daemon-w7n7w" (UID: "1d2712c7-d910-4a49-900d-ae83e771b3c1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:31.829732 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:31.829692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:31.829914 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:31.829748 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:31.830112 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:31.829926 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:31.830112 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:31.829988 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret podName:7c43931a-91df-49ea-bea3-10e907fe1b5c nodeName:}" failed. No retries permitted until 2026-07-15 16:58:35.829968743 +0000 UTC m=+10.143673617 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret") pod "global-pull-secret-syncer-d48rf" (UID: "7c43931a-91df-49ea-bea3-10e907fe1b5c") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:31.830559 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:31.830374 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:58:31.830559 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:31.830394 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:58:31.830559 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:31.830406 2566 projected.go:196] Error preparing data for projected volume kube-api-access-zbnxw for pod openshift-network-diagnostics/network-check-target-m4wq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:31.830559 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:31.830447 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw podName:fc473a95-bf8d-46ac-b047-8be9d1c1efc3 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:35.830433204 +0000 UTC m=+10.144138078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbnxw" (UniqueName: "kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw") pod "network-check-target-m4wq5" (UID: "fc473a95-bf8d-46ac-b047-8be9d1c1efc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:32.182683 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:32.182623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:32.182848 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:32.182694 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:32.182848 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:32.182661 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:32.182848 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:32.182798 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:32.183011 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:32.182893 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:32.183011 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:32.182990 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:34.184257 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:34.183741 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:34.184257 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:34.183785 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:34.184257 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:34.183750 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:34.184257 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:34.183888 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:34.184257 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:34.183969 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:34.184257 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:34.184021 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:35.766640 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:35.766585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:35.767236 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:35.766725 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:35.767236 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:35.766792 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs podName:1d2712c7-d910-4a49-900d-ae83e771b3c1 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:43.766773198 +0000 UTC m=+18.080478084 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs") pod "network-metrics-daemon-w7n7w" (UID: "1d2712c7-d910-4a49-900d-ae83e771b3c1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:35.867976 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:35.867937 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:35.868160 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:35.867988 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:35.868160 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:35.868108 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:35.868287 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:35.868168 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret podName:7c43931a-91df-49ea-bea3-10e907fe1b5c nodeName:}" failed. No retries permitted until 2026-07-15 16:58:43.868153962 +0000 UTC m=+18.181858852 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret") pod "global-pull-secret-syncer-d48rf" (UID: "7c43931a-91df-49ea-bea3-10e907fe1b5c") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:35.868287 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:35.868240 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:58:35.868287 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:35.868263 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:58:35.868287 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:35.868272 2566 projected.go:196] Error preparing data for projected volume kube-api-access-zbnxw for pod openshift-network-diagnostics/network-check-target-m4wq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:35.868475 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:35.868303 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw podName:fc473a95-bf8d-46ac-b047-8be9d1c1efc3 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:43.86829354 +0000 UTC m=+18.181998410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbnxw" (UniqueName: "kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw") pod "network-check-target-m4wq5" (UID: "fc473a95-bf8d-46ac-b047-8be9d1c1efc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:36.183737 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:36.183700 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:36.183926 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:36.183831 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:36.185265 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:36.185243 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:36.185378 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:36.185326 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:36.185378 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:36.185373 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:36.185481 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:36.185431 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:38.182948 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:38.182384 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:38.182948 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:38.182534 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:38.182948 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:38.182557 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:38.182948 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:38.182384 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:38.182948 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:38.182677 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:38.182948 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:38.182770 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:40.182253 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:40.182222 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:40.182649 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:40.182222 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:40.182649 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:40.182353 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:40.182649 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:40.182222 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:40.182649 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:40.182484 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:40.182649 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:40.182409 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:42.182091 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:42.182057 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:42.182564 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:42.182099 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:42.182564 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:42.182063 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:42.182564 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:42.182190 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:42.182564 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:42.182308 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:42.182564 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:42.182388 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:43.835897 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:43.835856 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:43.836294 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:43.836031 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:43.836294 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:43.836112 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs podName:1d2712c7-d910-4a49-900d-ae83e771b3c1 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:59.836090514 +0000 UTC m=+34.149795388 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs") pod "network-metrics-daemon-w7n7w" (UID: "1d2712c7-d910-4a49-900d-ae83e771b3c1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:43.936798 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:43.936766 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:43.936798 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:43.936803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:43.937003 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:43.936932 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:43.937003 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:43.936936 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:58:43.937003 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:43.936956 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:58:43.937003 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:43.936968 2566 projected.go:196] Error preparing data for projected volume kube-api-access-zbnxw for pod openshift-network-diagnostics/network-check-target-m4wq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:43.937003 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:43.936998 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret podName:7c43931a-91df-49ea-bea3-10e907fe1b5c nodeName:}" failed. No retries permitted until 2026-07-15 16:58:59.936981062 +0000 UTC m=+34.250685932 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret") pod "global-pull-secret-syncer-d48rf" (UID: "7c43931a-91df-49ea-bea3-10e907fe1b5c") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:43.937199 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:43.937015 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw podName:fc473a95-bf8d-46ac-b047-8be9d1c1efc3 nodeName:}" failed. No retries permitted until 2026-07-15 16:58:59.937008963 +0000 UTC m=+34.250713833 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbnxw" (UniqueName: "kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw") pod "network-check-target-m4wq5" (UID: "fc473a95-bf8d-46ac-b047-8be9d1c1efc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:44.182074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:44.181993 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:44.182074 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:44.182019 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:44.182293 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:44.181993 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:44.182293 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:44.182115 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:44.182293 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:44.182168 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:44.182293 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:44.182275 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:46.183089 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:46.183053 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:46.183548 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:46.183156 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:46.183548 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:46.183185 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:46.183548 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:46.183214 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:46.183548 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:46.183260 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:46.183548 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:46.183329 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:48.182783 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:48.182740 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:48.183485 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:48.182740 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:48.183485 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:48.182888 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:48.183485 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:48.182924 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:48.183485 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:48.182740 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:48.183485 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:48.183001 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:49.410078 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.409833 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-598cx" event={"ID":"c7342b5e-c845-43f4-9298-fd94f90fdc8a","Type":"ContainerStarted","Data":"36b96bb62508eda7ee715106538a65e0d5d87657ec76e542d51de6c6b6945fba"} Jul 15 16:58:49.411379 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.411357 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" event={"ID":"f442938c-8777-4cb5-ab3e-1f5dfaef9248","Type":"ContainerStarted","Data":"2f83403e4d7db6fa0e0a653956805a4589e1e93903d50d84a05cfc2077c752de"} Jul 15 16:58:49.412730 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.412708 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t8xnk" event={"ID":"2d7dc39f-d2fc-42e9-9338-decd2e825747","Type":"ContainerStarted","Data":"2134af0e11033893e5ffbd3b2bcebf3a65ffded8c2369a1d0a5cd3e15b575678"} Jul 15 16:58:49.414084 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.414060 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2pxcv" event={"ID":"25eeab2a-5a16-4d82-a085-22f22a60c078","Type":"ContainerStarted","Data":"f7dbd154c0ee144e636566263bc10d0fdaa4af830036329b185b982ba457e5e4"} Jul 15 16:58:49.415659 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.415626 2566 generic.go:356] "Generic (PLEG): container finished" podID="5d65117a-a7e7-4feb-9a0f-f05010d0c737" containerID="e02bfe93ef9425aa2758956fe034c20a73231f86d01bcc7207a9ea4c7653a2c0" exitCode=0 Jul 15 16:58:49.415771 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.415711 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" event={"ID":"5d65117a-a7e7-4feb-9a0f-f05010d0c737","Type":"ContainerDied","Data":"e02bfe93ef9425aa2758956fe034c20a73231f86d01bcc7207a9ea4c7653a2c0"} Jul 15 16:58:49.419268 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.419217 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w9zmt" event={"ID":"56812b13-8b82-41aa-83a9-0f5cf373ddfa","Type":"ContainerStarted","Data":"ad19c449bcd8e07c99b8ee40d0339a52bcc9cc3387ff5499c897e946b3d31afd"} Jul 15 16:58:49.422637 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.422596 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 16:58:49.422954 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.422935 2566 generic.go:356] "Generic (PLEG): container finished" podID="027a10e2-2eaa-42fa-8d0c-ceb61f74a71d" containerID="1c1d964c22f5dbefa39125991b696068b0c47a216c8ea67a8dcba1885a76afaa" exitCode=1 Jul 15 16:58:49.423017 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.423000 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerStarted","Data":"cfdc781a006c39dfb64069dd7e98afbb56026e6438e00d346498768165bed728"} Jul 15 16:58:49.423093 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.423066 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerStarted","Data":"2fa8fe6aafa49c8642c2023dfb48a40fd38c11e28c5edf820ffa5faf5ed2b104"} Jul 15 16:58:49.423194 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.423116 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerStarted","Data":"99952de4be3c6af1caadd749f3b1e0bd01381d4dd6d5dfcb159250f36f7b53a9"} Jul 15 16:58:49.423194 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.423161 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerStarted","Data":"7b9998279d7d9bc6b34524721abbccd877395b9e3e6d2c8e23b697a6373f7c56"} Jul 15 16:58:49.423289 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.423198 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerDied","Data":"1c1d964c22f5dbefa39125991b696068b0c47a216c8ea67a8dcba1885a76afaa"} Jul 15 16:58:49.423289 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.423247 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerStarted","Data":"c7fa634e64bf210a1006e3675e38e9f5a9ad1314bd8b2ac9e5825f41f6269742"} Jul 15 16:58:49.424366 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.424347 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lbkks" event={"ID":"aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7","Type":"ContainerStarted","Data":"bb37fedb36c8fe1c95abf7483887feb32a01c822da872c2e1122900eda13ce1a"} Jul 15 16:58:49.431726 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.431686 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-598cx" podStartSLOduration=4.125565519 podStartE2EDuration="23.431673919s" totalImagesPullingTime="19.3061084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:49.431215525 +0000 UTC m=+23.744920419" watchObservedRunningTime="2026-07-15 16:58:49.431673919 +0000 UTC m=+23.745378811" Jul 15 16:58:49.480357 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.480274 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t8xnk" podStartSLOduration=4.175800151 podStartE2EDuration="23.480255465s" totalImagesPullingTime="19.304455314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:49.448805217 +0000 UTC m=+23.762510108" watchObservedRunningTime="2026-07-15 16:58:49.480255465 +0000 UTC m=+23.793960358" Jul 15 16:58:49.480555 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.480523 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w9zmt" podStartSLOduration=4.189355606 podStartE2EDuration="23.480511307s" totalImagesPullingTime="19.291155701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:49.480171596 +0000 UTC m=+23.793876488" watchObservedRunningTime="2026-07-15 16:58:49.480511307 +0000 UTC m=+23.794216199" Jul 15 16:58:49.509151 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.509096 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2pxcv" podStartSLOduration=4.203394778 podStartE2EDuration="23.509081579s" totalImagesPullingTime="19.305686801s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:49.508501395 +0000 UTC m=+23.822206286" watchObservedRunningTime="2026-07-15 16:58:49.509081579 +0000 UTC m=+23.822786470" Jul 15 16:58:49.559175 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:49.559065 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lbkks" podStartSLOduration=4.203374678 podStartE2EDuration="23.55904508s" totalImagesPullingTime="19.355670402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:49.558399758 +0000 UTC m=+23.872104651" watchObservedRunningTime="2026-07-15 16:58:49.55904508 +0000 UTC m=+23.872749972" Jul 15 16:58:50.182367 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:50.182332 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:50.182367 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:50.182355 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:50.182699 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:50.182343 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:50.182699 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:50.182456 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:50.182699 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:50.182536 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:50.182699 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:50.182663 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:50.401452 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:50.401405 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 16:58:50.429618 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:50.429559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" event={"ID":"f442938c-8777-4cb5-ab3e-1f5dfaef9248","Type":"ContainerStarted","Data":"adb8535898e4b6b9bf1fbc66742685abbfb07a0f47fa145c82fca5b2ee541049"} Jul 15 16:58:50.432056 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:50.431984 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4rm7b" event={"ID":"dd5089e4-2941-42c5-9203-d6d3d7319ffc","Type":"ContainerStarted","Data":"ed6f5f33254578eabb9e2e05f3fb7d06ba0084c26791b02323abeab14be58f6e"} Jul 15 16:58:50.449783 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:50.449699 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4rm7b" podStartSLOduration=5.141946138 podStartE2EDuration="24.449682537s" totalImagesPullingTime="19.307736399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:50.4492107 +0000 UTC m=+24.762915709" watchObservedRunningTime="2026-07-15 16:58:50.449682537 +0000 UTC m=+24.763387429" Jul 15 16:58:51.147213 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:51.147080 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T16:58:50.401429294Z","UUID":"bb2ae0bb-9e21-4d0b-9891-b19a5a0ab0a4","Handler":null,"Name":"","Endpoint":""} Jul 15 16:58:51.148725 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:51.148699 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 16:58:51.148844 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:51.148737 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 16:58:52.182785 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:52.182579 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:52.183364 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:52.182888 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:52.183364 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:52.183252 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:52.183364 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:52.183346 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:52.183542 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:52.183437 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:52.183542 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:52.183514 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:52.441512 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:52.441443 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 16:58:52.441927 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:52.441886 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerStarted","Data":"045fd3f68dacf7fd53c811d8940c6da85fbc6ae40dffc53ccd9646c5afb70a5f"} Jul 15 16:58:52.444134 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:52.444110 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" event={"ID":"f442938c-8777-4cb5-ab3e-1f5dfaef9248","Type":"ContainerStarted","Data":"e823131c18bcd3607de2e77feecfed1cc902c2d410c3e19da7531d7a384482a6"} Jul 15 16:58:52.465424 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:52.465372 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5f2xd" podStartSLOduration=4.394171766 podStartE2EDuration="26.465356223s" totalImagesPullingTime="22.071184457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:52.463885849 +0000 UTC m=+26.777590744" watchObservedRunningTime="2026-07-15 16:58:52.465356223 +0000 UTC m=+26.779061115" Jul 15 16:58:53.620424 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:53.620386 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:53.621194 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:53.621039 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:54.182476 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.182305 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:54.182627 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.182309 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:54.182627 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:54.182582 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:54.182627 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.182309 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:54.182791 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:54.182661 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:54.182791 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:54.182687 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:54.451412 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.451327 2566 generic.go:356] "Generic (PLEG): container finished" podID="5d65117a-a7e7-4feb-9a0f-f05010d0c737" containerID="877d2490fb2fa6a9510a76191bc21c608443ab67e84bfe85f72e05dbe62ab73e" exitCode=0 Jul 15 16:58:54.451558 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.451418 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" event={"ID":"5d65117a-a7e7-4feb-9a0f-f05010d0c737","Type":"ContainerDied","Data":"877d2490fb2fa6a9510a76191bc21c608443ab67e84bfe85f72e05dbe62ab73e"} Jul 15 16:58:54.454894 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.454874 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 16:58:54.455256 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.455234 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerStarted","Data":"05e45672d3615dad3e38be8444e8254593a07f093a337a6c94bac7582bbe18a0"} Jul 15 16:58:54.455804 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.455597 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:54.455804 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.455692 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:54.455804 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.455729 2566 scope.go:117] "RemoveContainer" containerID="1c1d964c22f5dbefa39125991b696068b0c47a216c8ea67a8dcba1885a76afaa" Jul 15 16:58:54.455804 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.455737 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:54.455804 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.455754 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:54.456457 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.456442 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w9zmt" Jul 15 16:58:54.472168 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.472140 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:54.472789 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:54.472773 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:58:55.461518 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:55.461490 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 16:58:55.462162 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:55.461911 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" event={"ID":"027a10e2-2eaa-42fa-8d0c-ceb61f74a71d","Type":"ContainerStarted","Data":"fc3a8eebb20876de7e22c85d8bcdc60c99f7b7177957e89dea81f5466d4b1938"} Jul 15 16:58:55.498318 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:55.498260 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" podStartSLOduration=10.126869057 podStartE2EDuration="29.498239864s" totalImagesPullingTime="19.371370807s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:58:55.496766763 +0000 UTC m=+29.810471654" watchObservedRunningTime="2026-07-15 16:58:55.498239864 +0000 UTC m=+29.811944758" Jul 15 16:58:56.014330 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:56.014299 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m4wq5"] Jul 15 16:58:56.014553 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:56.014418 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:56.014553 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:56.014502 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:56.017585 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:56.017555 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d48rf"] Jul 15 16:58:56.017735 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:56.017675 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:56.017796 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:56.017766 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:56.018348 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:56.018321 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w7n7w"] Jul 15 16:58:56.018448 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:56.018436 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:56.018563 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:56.018544 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:56.465880 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:56.465684 2566 generic.go:356] "Generic (PLEG): container finished" podID="5d65117a-a7e7-4feb-9a0f-f05010d0c737" containerID="cbbb0b092ab5af41fc0cd0b3144ea431bd08c9e8be2bee2aa8681e4f4c939649" exitCode=0 Jul 15 16:58:56.466311 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:56.465764 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" event={"ID":"5d65117a-a7e7-4feb-9a0f-f05010d0c737","Type":"ContainerDied","Data":"cbbb0b092ab5af41fc0cd0b3144ea431bd08c9e8be2bee2aa8681e4f4c939649"} Jul 15 16:58:57.182419 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:57.182379 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:57.182640 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:57.182500 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:58.182265 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:58.182236 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:58.182265 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:58.182254 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:58.182947 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:58.182392 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:58:58.182947 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:58.182503 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:58:58.472689 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:58.472616 2566 generic.go:356] "Generic (PLEG): container finished" podID="5d65117a-a7e7-4feb-9a0f-f05010d0c737" containerID="55442705594d42cd2ba576786b3f6caa151f9e4847b65242747c450a140d0829" exitCode=0 Jul 15 16:58:58.472689 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:58.472638 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" event={"ID":"5d65117a-a7e7-4feb-9a0f-f05010d0c737","Type":"ContainerDied","Data":"55442705594d42cd2ba576786b3f6caa151f9e4847b65242747c450a140d0829"} Jul 15 16:58:59.181909 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:59.181882 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:59.182085 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:59.181986 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:58:59.863799 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:59.863761 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:58:59.864328 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:59.863888 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:59.864328 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:59.863953 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs podName:1d2712c7-d910-4a49-900d-ae83e771b3c1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:31.863932581 +0000 UTC m=+66.177637458 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs") pod "network-metrics-daemon-w7n7w" (UID: "1d2712c7-d910-4a49-900d-ae83e771b3c1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:58:59.964732 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:59.964699 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:58:59.964732 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:58:59.964742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:58:59.964981 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:59.964880 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:59.964981 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:59.964940 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret podName:7c43931a-91df-49ea-bea3-10e907fe1b5c nodeName:}" failed. No retries permitted until 2026-07-15 16:59:31.964923445 +0000 UTC m=+66.278628335 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret") pod "global-pull-secret-syncer-d48rf" (UID: "7c43931a-91df-49ea-bea3-10e907fe1b5c") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:58:59.965357 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:59.965336 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:58:59.965357 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:59.965357 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:58:59.965527 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:59.965369 2566 projected.go:196] Error preparing data for projected volume kube-api-access-zbnxw for pod openshift-network-diagnostics/network-check-target-m4wq5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:58:59.965527 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:58:59.965412 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw podName:fc473a95-bf8d-46ac-b047-8be9d1c1efc3 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:31.965398168 +0000 UTC m=+66.279103249 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zbnxw" (UniqueName: "kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw") pod "network-check-target-m4wq5" (UID: "fc473a95-bf8d-46ac-b047-8be9d1c1efc3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:59:00.182757 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:00.182673 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:59:00.182918 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:00.182683 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:59:00.182918 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:00.182811 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d48rf" podUID="7c43931a-91df-49ea-bea3-10e907fe1b5c" Jul 15 16:59:00.182918 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:00.182874 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w7n7w" podUID="1d2712c7-d910-4a49-900d-ae83e771b3c1" Jul 15 16:59:01.182010 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.181972 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:59:01.182384 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.182104 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m4wq5" podUID="fc473a95-bf8d-46ac-b047-8be9d1c1efc3" Jul 15 16:59:01.518879 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.518799 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-124.ec2.internal" event="NodeReady" Jul 15 16:59:01.519042 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.518939 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 16:59:01.561208 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.561178 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-756d85f79-ct589"] Jul 15 16:59:01.563708 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.563683 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.569265 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.568869 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 16:59:01.569265 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.569099 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp"] Jul 15 16:59:01.569265 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.569148 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9d5w4\"" Jul 15 16:59:01.570083 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.570058 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 16:59:01.570320 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.570289 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 16:59:01.571096 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.571076 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:01.573982 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.573961 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-nzcjg"] Jul 15 16:59:01.575803 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.575771 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5"] Jul 15 16:59:01.576484 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.576455 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 16:59:01.576724 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.576680 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 16:59:01.577479 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.576907 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.577479 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.576948 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4ghtm\"" Jul 15 16:59:01.577479 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.577169 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 16:59:01.578744 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.578724 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 16:59:01.580001 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.579977 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 16:59:01.580676 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.580590 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 16:59:01.582748 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.580982 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7d99b5d79b-cfd7r"] Jul 15 16:59:01.582748 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.581191 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 16:59:01.582748 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.581244 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.582748 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.581246 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 15 16:59:01.582748 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.581754 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 15 16:59:01.582748 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.582068 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lt77r\"" Jul 15 16:59:01.584244 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.584225 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.584972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.584950 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 16:59:01.585284 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.585264 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 16:59:01.585373 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.585320 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:59:01.585423 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.585374 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-9gmcp\"" Jul 15 16:59:01.585423 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.585325 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 16:59:01.587908 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.587887 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 15 16:59:01.592259 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.592237 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xvxm7\"" Jul 15 16:59:01.593371 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.592580 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 16:59:01.593371 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.592590 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 16:59:01.593371 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.592596 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 16:59:01.593371 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.592755 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 16:59:01.593371 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.592760 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 16:59:01.593371 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.593011 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 16:59:01.593925 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.593903 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-756d85f79-ct589"] Jul 15 16:59:01.595093 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.595076 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jht7b"] Jul 15 16:59:01.602426 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.602397 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp"] Jul 15 16:59:01.602552 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.602443 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5"] Jul 15 16:59:01.602552 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.602470 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v78pr"] Jul 15 16:59:01.602796 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.602777 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.605102 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.605082 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-nzcjg"] Jul 15 16:59:01.605356 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.605331 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v78pr" Jul 15 16:59:01.605936 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.605915 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 16:59:01.606033 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.605937 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 16:59:01.606033 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.606012 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5hxs5\"" Jul 15 16:59:01.607977 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.607955 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 16:59:01.608471 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.608099 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 16:59:01.608471 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.608290 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kt5f5\"" Jul 15 16:59:01.608471 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.608316 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 16:59:01.610910 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.610890 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jht7b"] Jul 15 16:59:01.615397 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.615378 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d99b5d79b-cfd7r"] Jul 15 16:59:01.616317 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.616295 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v78pr"] Jul 15 16:59:01.679776 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.679743 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63bf6178-a18e-4b9c-81bb-98a008e87fc8-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.679776 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.679775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63bf6178-a18e-4b9c-81bb-98a008e87fc8-serving-cert\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.679776 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.679795 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrmvl\" (UniqueName: \"kubernetes.io/projected/63bf6178-a18e-4b9c-81bb-98a008e87fc8-kube-api-access-lrmvl\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.680092 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.679811 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c975720-099d-446b-bcf7-cb34eb853f8a-config\") pod \"service-ca-operator-7695fc6c8c-8z7k5\" (UID: \"1c975720-099d-446b-bcf7-cb34eb853f8a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.680092 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.679834 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63bf6178-a18e-4b9c-81bb-98a008e87fc8-tmp\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.680092 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.679946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.680092 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.679980 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:01.680092 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.679998 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76qqp\" (UniqueName: \"kubernetes.io/projected/8c9266b8-1422-4e7a-9736-89316363fc85-kube-api-access-76qqp\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:01.680092 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680021 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbr9k\" (UniqueName: \"kubernetes.io/projected/1c975720-099d-446b-bcf7-cb34eb853f8a-kube-api-access-qbr9k\") pod \"service-ca-operator-7695fc6c8c-8z7k5\" (UID: \"1c975720-099d-446b-bcf7-cb34eb853f8a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.680092 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680048 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-stats-auth\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680103 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-certificates\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c975720-099d-446b-bcf7-cb34eb853f8a-serving-cert\") pod \"service-ca-operator-7695fc6c8c-8z7k5\" (UID: \"1c975720-099d-446b-bcf7-cb34eb853f8a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680143 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjhnl\" (UniqueName: \"kubernetes.io/projected/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-kube-api-access-mjhnl\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680183 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-default-certificate\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680205 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-image-registry-private-configuration\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-trusted-ca\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680259 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/63bf6178-a18e-4b9c-81bb-98a008e87fc8-snapshots\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680277 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-installation-pull-secrets\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-ca-trust-extracted\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzlhh\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-kube-api-access-kzlhh\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680370 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-bound-sa-token\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680387 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63bf6178-a18e-4b9c-81bb-98a008e87fc8-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680402 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.680434 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.680421 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8c9266b8-1422-4e7a-9736-89316363fc85-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:01.781535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781447 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f958\" (UniqueName: \"kubernetes.io/projected/30da06ef-d2dd-4cdf-bcfa-02bf28378483-kube-api-access-7f958\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.781535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781490 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.781535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781508 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.781535 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781530 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-bound-sa-token\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.781783 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.781783 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.781648 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:59:01.781783 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.781667 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-756d85f79-ct589: secret "image-registry-tls" not found Jul 15 16:59:01.781783 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.781703 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:02.281682238 +0000 UTC m=+36.595387131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : configmap references non-existent config key: service-ca.crt Jul 15 16:59:01.781952 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.781780 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls podName:83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:02.281763867 +0000 UTC m=+36.595468741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls") pod "image-registry-756d85f79-ct589" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2") : secret "image-registry-tls" not found Jul 15 16:59:01.781952 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781806 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8c9266b8-1422-4e7a-9736-89316363fc85-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:01.781952 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781843 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63bf6178-a18e-4b9c-81bb-98a008e87fc8-tmp\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.781952 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781869 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stg9w\" (UniqueName: \"kubernetes.io/projected/3ee02540-b301-416f-9ac2-41c5170668c4-kube-api-access-stg9w\") pod \"ingress-canary-v78pr\" (UID: \"3ee02540-b301-416f-9ac2-41c5170668c4\") " pod="openshift-ingress-canary/ingress-canary-v78pr" Jul 15 16:59:01.781952 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781904 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-installation-pull-secrets\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.781952 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.781934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c975720-099d-446b-bcf7-cb34eb853f8a-config\") pod \"service-ca-operator-7695fc6c8c-8z7k5\" (UID: \"1c975720-099d-446b-bcf7-cb34eb853f8a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.782230 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782034 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ee02540-b301-416f-9ac2-41c5170668c4-cert\") pod \"ingress-canary-v78pr\" (UID: \"3ee02540-b301-416f-9ac2-41c5170668c4\") " pod="openshift-ingress-canary/ingress-canary-v78pr" Jul 15 16:59:01.782230 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782070 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-stats-auth\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.782230 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782107 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.782230 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782132 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-ca-trust-extracted\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.782230 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782159 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:01.782230 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782188 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbr9k\" (UniqueName: \"kubernetes.io/projected/1c975720-099d-446b-bcf7-cb34eb853f8a-kube-api-access-qbr9k\") pod \"service-ca-operator-7695fc6c8c-8z7k5\" (UID: \"1c975720-099d-446b-bcf7-cb34eb853f8a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.782461 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782234 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63bf6178-a18e-4b9c-81bb-98a008e87fc8-tmp\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.782461 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782237 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-certificates\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.782461 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782264 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/30da06ef-d2dd-4cdf-bcfa-02bf28378483-config-volume\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.782714 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782691 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1c975720-099d-446b-bcf7-cb34eb853f8a-config\") pod \"service-ca-operator-7695fc6c8c-8z7k5\" (UID: \"1c975720-099d-446b-bcf7-cb34eb853f8a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.782866 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782724 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/8c9266b8-1422-4e7a-9736-89316363fc85-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:01.782866 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.782817 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:59:01.782978 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782871 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-certificates\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.782978 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.782884 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:02.282866875 +0000 UTC m=+36.596571748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : secret "router-metrics-certs-default" not found Jul 15 16:59:01.782978 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782905 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-ca-trust-extracted\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.782978 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.782945 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:01.782978 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.782969 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-trusted-ca\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.782978 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.782981 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls podName:8c9266b8-1422-4e7a-9736-89316363fc85 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:02.282969474 +0000 UTC m=+36.596674347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wd9bp" (UID: "8c9266b8-1422-4e7a-9736-89316363fc85") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:01.783178 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783019 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/63bf6178-a18e-4b9c-81bb-98a008e87fc8-snapshots\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.783226 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783205 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63bf6178-a18e-4b9c-81bb-98a008e87fc8-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.783300 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783243 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63bf6178-a18e-4b9c-81bb-98a008e87fc8-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.783345 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783320 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63bf6178-a18e-4b9c-81bb-98a008e87fc8-serving-cert\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.783388 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783344 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrmvl\" (UniqueName: \"kubernetes.io/projected/63bf6178-a18e-4b9c-81bb-98a008e87fc8-kube-api-access-lrmvl\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.783388 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783378 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzlhh\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-kube-api-access-kzlhh\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.783464 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783411 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76qqp\" (UniqueName: \"kubernetes.io/projected/8c9266b8-1422-4e7a-9736-89316363fc85-kube-api-access-76qqp\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:01.783464 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c975720-099d-446b-bcf7-cb34eb853f8a-serving-cert\") pod \"service-ca-operator-7695fc6c8c-8z7k5\" (UID: \"1c975720-099d-446b-bcf7-cb34eb853f8a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.783526 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783475 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/63bf6178-a18e-4b9c-81bb-98a008e87fc8-snapshots\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.783526 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783478 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjhnl\" (UniqueName: \"kubernetes.io/projected/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-kube-api-access-mjhnl\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.783590 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783527 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-default-certificate\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.783590 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783553 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-image-registry-private-configuration\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.783590 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783581 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30da06ef-d2dd-4cdf-bcfa-02bf28378483-tmp-dir\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.783892 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.783868 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-trusted-ca\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.785529 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.785291 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63bf6178-a18e-4b9c-81bb-98a008e87fc8-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.785791 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.785598 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63bf6178-a18e-4b9c-81bb-98a008e87fc8-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.790835 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.787491 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-stats-auth\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.790835 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.789470 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-default-certificate\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.791687 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.791625 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-installation-pull-secrets\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.792055 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.792006 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/63bf6178-a18e-4b9c-81bb-98a008e87fc8-serving-cert\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.793356 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.792896 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1c975720-099d-446b-bcf7-cb34eb853f8a-serving-cert\") pod \"service-ca-operator-7695fc6c8c-8z7k5\" (UID: \"1c975720-099d-446b-bcf7-cb34eb853f8a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.793356 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.792981 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-image-registry-private-configuration\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.794081 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.794041 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbr9k\" (UniqueName: \"kubernetes.io/projected/1c975720-099d-446b-bcf7-cb34eb853f8a-kube-api-access-qbr9k\") pod \"service-ca-operator-7695fc6c8c-8z7k5\" (UID: \"1c975720-099d-446b-bcf7-cb34eb853f8a\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.794444 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.794418 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrmvl\" (UniqueName: \"kubernetes.io/projected/63bf6178-a18e-4b9c-81bb-98a008e87fc8-kube-api-access-lrmvl\") pod \"insights-operator-68d9dd94cf-nzcjg\" (UID: \"63bf6178-a18e-4b9c-81bb-98a008e87fc8\") " pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.794548 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.794525 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzlhh\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-kube-api-access-kzlhh\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.795647 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.795593 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76qqp\" (UniqueName: \"kubernetes.io/projected/8c9266b8-1422-4e7a-9736-89316363fc85-kube-api-access-76qqp\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:01.795946 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.795927 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-bound-sa-token\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:01.796828 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.796798 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjhnl\" (UniqueName: \"kubernetes.io/projected/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-kube-api-access-mjhnl\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:01.884739 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.884685 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30da06ef-d2dd-4cdf-bcfa-02bf28378483-tmp-dir\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.884929 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.884766 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f958\" (UniqueName: \"kubernetes.io/projected/30da06ef-d2dd-4cdf-bcfa-02bf28378483-kube-api-access-7f958\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.884929 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.884807 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.884929 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.884852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stg9w\" (UniqueName: \"kubernetes.io/projected/3ee02540-b301-416f-9ac2-41c5170668c4-kube-api-access-stg9w\") pod \"ingress-canary-v78pr\" (UID: \"3ee02540-b301-416f-9ac2-41c5170668c4\") " pod="openshift-ingress-canary/ingress-canary-v78pr" Jul 15 16:59:01.884929 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.884883 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ee02540-b301-416f-9ac2-41c5170668c4-cert\") pod \"ingress-canary-v78pr\" (UID: \"3ee02540-b301-416f-9ac2-41c5170668c4\") " pod="openshift-ingress-canary/ingress-canary-v78pr" Jul 15 16:59:01.885132 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.884965 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:59:01.885132 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:01.885036 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls podName:30da06ef-d2dd-4cdf-bcfa-02bf28378483 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:02.385016402 +0000 UTC m=+36.698721272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls") pod "dns-default-jht7b" (UID: "30da06ef-d2dd-4cdf-bcfa-02bf28378483") : secret "dns-default-metrics-tls" not found Jul 15 16:59:01.885223 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.885127 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30da06ef-d2dd-4cdf-bcfa-02bf28378483-tmp-dir\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.885223 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.885201 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/30da06ef-d2dd-4cdf-bcfa-02bf28378483-config-volume\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.885731 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.885708 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/30da06ef-d2dd-4cdf-bcfa-02bf28378483-config-volume\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.889857 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.889823 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3ee02540-b301-416f-9ac2-41c5170668c4-cert\") pod \"ingress-canary-v78pr\" (UID: \"3ee02540-b301-416f-9ac2-41c5170668c4\") " pod="openshift-ingress-canary/ingress-canary-v78pr" Jul 15 16:59:01.895793 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.895762 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f958\" (UniqueName: \"kubernetes.io/projected/30da06ef-d2dd-4cdf-bcfa-02bf28378483-kube-api-access-7f958\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:01.895945 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.895821 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stg9w\" (UniqueName: \"kubernetes.io/projected/3ee02540-b301-416f-9ac2-41c5170668c4-kube-api-access-stg9w\") pod \"ingress-canary-v78pr\" (UID: \"3ee02540-b301-416f-9ac2-41c5170668c4\") " pod="openshift-ingress-canary/ingress-canary-v78pr" Jul 15 16:59:01.896539 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.896520 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" Jul 15 16:59:01.904290 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.904268 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" Jul 15 16:59:01.929578 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:01.929540 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v78pr" Jul 15 16:59:02.182029 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.181990 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:59:02.182504 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.182211 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:59:02.186192 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.186167 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 16:59:02.186342 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.186207 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 16:59:02.187239 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.187216 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tns8n\"" Jul 15 16:59:02.288489 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.288452 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:02.288683 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.288515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:02.288683 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.288645 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:59:02.288683 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.288655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:02.288796 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.288698 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:03.288680612 +0000 UTC m=+37.602385481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : configmap references non-existent config key: service-ca.crt Jul 15 16:59:02.288796 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.288730 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:02.288796 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.288766 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls podName:8c9266b8-1422-4e7a-9736-89316363fc85 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:03.288750374 +0000 UTC m=+37.602455262 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wd9bp" (UID: "8c9266b8-1422-4e7a-9736-89316363fc85") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:02.288796 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.288786 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:03.288777259 +0000 UTC m=+37.602482135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : secret "router-metrics-certs-default" not found Jul 15 16:59:02.288946 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.288874 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:02.289009 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.288988 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:59:02.289009 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.289009 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-756d85f79-ct589: secret "image-registry-tls" not found Jul 15 16:59:02.289188 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.289054 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls podName:83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:03.289043152 +0000 UTC m=+37.602748044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls") pod "image-registry-756d85f79-ct589" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2") : secret "image-registry-tls" not found Jul 15 16:59:02.389972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.389926 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:02.390165 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.390083 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:59:02.390165 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:02.390162 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls podName:30da06ef-d2dd-4cdf-bcfa-02bf28378483 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:03.390145604 +0000 UTC m=+37.703850475 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls") pod "dns-default-jht7b" (UID: "30da06ef-d2dd-4cdf-bcfa-02bf28378483") : secret "dns-default-metrics-tls" not found Jul 15 16:59:02.643181 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.643154 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5"] Jul 15 16:59:02.646797 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:02.646764 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c975720_099d_446b_bcf7_cb34eb853f8a.slice/crio-8fb07da167e8c8648190f5ddc730dd40479c97acc73abd73afe18c5f090c74ee WatchSource:0}: Error finding container 8fb07da167e8c8648190f5ddc730dd40479c97acc73abd73afe18c5f090c74ee: Status 404 returned error can't find the container with id 8fb07da167e8c8648190f5ddc730dd40479c97acc73abd73afe18c5f090c74ee Jul 15 16:59:02.651842 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.651814 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-nzcjg"] Jul 15 16:59:02.654782 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:02.654724 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v78pr"] Jul 15 16:59:02.659793 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:02.657885 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63bf6178_a18e_4b9c_81bb_98a008e87fc8.slice/crio-9ef716ef838cf9489aeb6929097dd41c24ac1aadf82a14bbba602f99f1e433ad WatchSource:0}: Error finding container 9ef716ef838cf9489aeb6929097dd41c24ac1aadf82a14bbba602f99f1e433ad: Status 404 returned error can't find the container with id 9ef716ef838cf9489aeb6929097dd41c24ac1aadf82a14bbba602f99f1e433ad Jul 15 16:59:02.659793 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:02.658594 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ee02540_b301_416f_9ac2_41c5170668c4.slice/crio-f41ee79c9e52de341050498f45db219563bf18b612ce81cd184c66e6a2ed373a WatchSource:0}: Error finding container f41ee79c9e52de341050498f45db219563bf18b612ce81cd184c66e6a2ed373a: Status 404 returned error can't find the container with id f41ee79c9e52de341050498f45db219563bf18b612ce81cd184c66e6a2ed373a Jul 15 16:59:03.181716 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.181677 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:59:03.185432 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.185405 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 16:59:03.185946 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.185749 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 16:59:03.186872 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.186852 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f7t54\"" Jul 15 16:59:03.298108 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.298071 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:03.298313 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.298139 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:03.298313 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.298172 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:03.298313 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.298264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:03.298464 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.298420 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:59:03.298464 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.298438 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-756d85f79-ct589: secret "image-registry-tls" not found Jul 15 16:59:03.298556 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.298496 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls podName:83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:05.29847745 +0000 UTC m=+39.612182335 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls") pod "image-registry-756d85f79-ct589" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2") : secret "image-registry-tls" not found Jul 15 16:59:03.298989 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.298971 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:05.298954064 +0000 UTC m=+39.612658936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : configmap references non-existent config key: service-ca.crt Jul 15 16:59:03.299087 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.299054 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:59:03.299149 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.299090 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:05.29907832 +0000 UTC m=+39.612783207 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : secret "router-metrics-certs-default" not found Jul 15 16:59:03.299207 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.299163 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:03.299207 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.299195 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls podName:8c9266b8-1422-4e7a-9736-89316363fc85 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:05.299184015 +0000 UTC m=+39.612888889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wd9bp" (UID: "8c9266b8-1422-4e7a-9736-89316363fc85") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:03.399708 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.399667 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:03.399910 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.399885 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:59:03.400032 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:03.399946 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls podName:30da06ef-d2dd-4cdf-bcfa-02bf28378483 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:05.399931419 +0000 UTC m=+39.713636290 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls") pod "dns-default-jht7b" (UID: "30da06ef-d2dd-4cdf-bcfa-02bf28378483") : secret "dns-default-metrics-tls" not found Jul 15 16:59:03.488513 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.488433 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v78pr" event={"ID":"3ee02540-b301-416f-9ac2-41c5170668c4","Type":"ContainerStarted","Data":"f41ee79c9e52de341050498f45db219563bf18b612ce81cd184c66e6a2ed373a"} Jul 15 16:59:03.491909 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.491869 2566 generic.go:356] "Generic (PLEG): container finished" podID="5d65117a-a7e7-4feb-9a0f-f05010d0c737" containerID="a5a9545586d4d5776e1c538cb464607f041544ceabac14757ff9d503e880af83" exitCode=0 Jul 15 16:59:03.492046 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.491963 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" event={"ID":"5d65117a-a7e7-4feb-9a0f-f05010d0c737","Type":"ContainerDied","Data":"a5a9545586d4d5776e1c538cb464607f041544ceabac14757ff9d503e880af83"} Jul 15 16:59:03.494911 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.494882 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" event={"ID":"1c975720-099d-446b-bcf7-cb34eb853f8a","Type":"ContainerStarted","Data":"8fb07da167e8c8648190f5ddc730dd40479c97acc73abd73afe18c5f090c74ee"} Jul 15 16:59:03.496711 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:03.496686 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" event={"ID":"63bf6178-a18e-4b9c-81bb-98a008e87fc8","Type":"ContainerStarted","Data":"9ef716ef838cf9489aeb6929097dd41c24ac1aadf82a14bbba602f99f1e433ad"} Jul 15 16:59:04.412555 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.412327 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd"] Jul 15 16:59:04.415010 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.414991 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd" Jul 15 16:59:04.418221 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.417965 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:59:04.418221 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.417992 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-t4r9h\"" Jul 15 16:59:04.419101 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.419080 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 16:59:04.428628 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.428585 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd"] Jul 15 16:59:04.491549 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.491515 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:04.504811 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.504777 2566 generic.go:356] "Generic (PLEG): container finished" podID="5d65117a-a7e7-4feb-9a0f-f05010d0c737" containerID="e8e0ccd2a7935174d9a777b20abec07aa5ee453d451a12af4b29e42e958d1d36" exitCode=0 Jul 15 16:59:04.504951 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.504848 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" event={"ID":"5d65117a-a7e7-4feb-9a0f-f05010d0c737","Type":"ContainerDied","Data":"e8e0ccd2a7935174d9a777b20abec07aa5ee453d451a12af4b29e42e958d1d36"} Jul 15 16:59:04.598696 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.598651 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb"] Jul 15 16:59:04.602731 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.602706 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-bb8gz"] Jul 15 16:59:04.602877 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.602843 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.605699 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.605496 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 15 16:59:04.605699 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.605554 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:59:04.605917 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.605901 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 15 16:59:04.606082 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.606060 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 15 16:59:04.606255 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.606238 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qmnm5\"" Jul 15 16:59:04.606838 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.606819 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.609024 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.608997 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl5zj\" (UniqueName: \"kubernetes.io/projected/596e6e61-c5fe-449a-833e-861d4588e9fb-kube-api-access-jl5zj\") pod \"volume-data-source-validator-78495597f6-gldzd\" (UID: \"596e6e61-c5fe-449a-833e-861d4588e9fb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd" Jul 15 16:59:04.609425 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.609410 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 15 16:59:04.609930 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.609731 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:59:04.609930 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.609740 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 15 16:59:04.610178 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.610160 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-kfjs2\"" Jul 15 16:59:04.612301 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.612156 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 15 16:59:04.618064 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.618041 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-bb8gz"] Jul 15 16:59:04.619133 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.619100 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb"] Jul 15 16:59:04.620327 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.620301 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 15 16:59:04.691445 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.691311 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf"] Jul 15 16:59:04.709645 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.709583 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:04.709645 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.709622 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8a3ef658-21e3-409d-973a-ef9afbbb3b46-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-qq9xb\" (UID: \"8a3ef658-21e3-409d-973a-ef9afbbb3b46\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.709888 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.709671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b68rl\" (UniqueName: \"kubernetes.io/projected/4f3aa3b2-8da9-47a5-a23e-72974a650bac-kube-api-access-b68rl\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.709888 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.709724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8a3ef658-21e3-409d-973a-ef9afbbb3b46-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-qq9xb\" (UID: \"8a3ef658-21e3-409d-973a-ef9afbbb3b46\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.709888 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.709746 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f3aa3b2-8da9-47a5-a23e-72974a650bac-config\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.709888 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.709812 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3aa3b2-8da9-47a5-a23e-72974a650bac-serving-cert\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.709888 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.709841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f3aa3b2-8da9-47a5-a23e-72974a650bac-trusted-ca\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.709888 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.709884 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl5zj\" (UniqueName: \"kubernetes.io/projected/596e6e61-c5fe-449a-833e-861d4588e9fb-kube-api-access-jl5zj\") pod \"volume-data-source-validator-78495597f6-gldzd\" (UID: \"596e6e61-c5fe-449a-833e-861d4588e9fb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd" Jul 15 16:59:04.710177 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.709931 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9p9r\" (UniqueName: \"kubernetes.io/projected/8a3ef658-21e3-409d-973a-ef9afbbb3b46-kube-api-access-g9p9r\") pod \"kube-storage-version-migrator-operator-799cb7f958-qq9xb\" (UID: \"8a3ef658-21e3-409d-973a-ef9afbbb3b46\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.712174 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.712146 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf"] Jul 15 16:59:04.713545 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.713522 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 15 16:59:04.713708 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.713533 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:59:04.713708 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.713597 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 15 16:59:04.713935 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.713914 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wk6nl\"" Jul 15 16:59:04.741911 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.741876 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl5zj\" (UniqueName: \"kubernetes.io/projected/596e6e61-c5fe-449a-833e-861d4588e9fb-kube-api-access-jl5zj\") pod \"volume-data-source-validator-78495597f6-gldzd\" (UID: \"596e6e61-c5fe-449a-833e-861d4588e9fb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd" Jul 15 16:59:04.767670 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.767632 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:04.810597 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.810562 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8a3ef658-21e3-409d-973a-ef9afbbb3b46-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-qq9xb\" (UID: \"8a3ef658-21e3-409d-973a-ef9afbbb3b46\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.810792 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.810630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b68rl\" (UniqueName: \"kubernetes.io/projected/4f3aa3b2-8da9-47a5-a23e-72974a650bac-kube-api-access-b68rl\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.810792 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.810688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8a3ef658-21e3-409d-973a-ef9afbbb3b46-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-qq9xb\" (UID: \"8a3ef658-21e3-409d-973a-ef9afbbb3b46\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.810792 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.810708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f3aa3b2-8da9-47a5-a23e-72974a650bac-config\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.810792 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.810736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:04.810792 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.810782 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3aa3b2-8da9-47a5-a23e-72974a650bac-serving-cert\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.811066 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.810806 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57ftd\" (UniqueName: \"kubernetes.io/projected/b504a6a8-6759-41b5-960f-83ffe042f01a-kube-api-access-57ftd\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:04.811066 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.810832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f3aa3b2-8da9-47a5-a23e-72974a650bac-trusted-ca\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.811066 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.810900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9p9r\" (UniqueName: \"kubernetes.io/projected/8a3ef658-21e3-409d-973a-ef9afbbb3b46-kube-api-access-g9p9r\") pod \"kube-storage-version-migrator-operator-799cb7f958-qq9xb\" (UID: \"8a3ef658-21e3-409d-973a-ef9afbbb3b46\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.811382 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.811345 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8a3ef658-21e3-409d-973a-ef9afbbb3b46-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-qq9xb\" (UID: \"8a3ef658-21e3-409d-973a-ef9afbbb3b46\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.811470 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.811435 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4f3aa3b2-8da9-47a5-a23e-72974a650bac-config\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.811555 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.811538 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f3aa3b2-8da9-47a5-a23e-72974a650bac-trusted-ca\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.813482 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.813462 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8a3ef658-21e3-409d-973a-ef9afbbb3b46-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-qq9xb\" (UID: \"8a3ef658-21e3-409d-973a-ef9afbbb3b46\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.813717 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.813695 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4f3aa3b2-8da9-47a5-a23e-72974a650bac-serving-cert\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.823328 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.823298 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b68rl\" (UniqueName: \"kubernetes.io/projected/4f3aa3b2-8da9-47a5-a23e-72974a650bac-kube-api-access-b68rl\") pod \"console-operator-7866497494-bb8gz\" (UID: \"4f3aa3b2-8da9-47a5-a23e-72974a650bac\") " pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.824030 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.824004 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9p9r\" (UniqueName: \"kubernetes.io/projected/8a3ef658-21e3-409d-973a-ef9afbbb3b46-kube-api-access-g9p9r\") pod \"kube-storage-version-migrator-operator-799cb7f958-qq9xb\" (UID: \"8a3ef658-21e3-409d-973a-ef9afbbb3b46\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.867888 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.867825 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:04.911391 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.911350 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:04.911554 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.911408 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57ftd\" (UniqueName: \"kubernetes.io/projected/b504a6a8-6759-41b5-960f-83ffe042f01a-kube-api-access-57ftd\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:04.911554 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:04.911502 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 16:59:04.911642 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:04.911566 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls podName:b504a6a8-6759-41b5-960f-83ffe042f01a nodeName:}" failed. No retries permitted until 2026-07-15 16:59:05.411549017 +0000 UTC m=+39.725253886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls") pod "cluster-samples-operator-65db858894-wp8hf" (UID: "b504a6a8-6759-41b5-960f-83ffe042f01a") : secret "samples-operator-tls" not found Jul 15 16:59:04.915387 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.915368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" Jul 15 16:59:04.926054 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.925935 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:04.927102 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:04.927080 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57ftd\" (UniqueName: \"kubernetes.io/projected/b504a6a8-6759-41b5-960f-83ffe042f01a-kube-api-access-57ftd\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:05.024399 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.024314 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd" Jul 15 16:59:05.314316 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.314239 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:05.314316 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.314297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:05.314533 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.314359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:05.314533 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.314386 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:05.314533 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.314414 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:59:05.314533 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.314436 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-756d85f79-ct589: secret "image-registry-tls" not found Jul 15 16:59:05.314533 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.314482 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:59:05.314533 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.314498 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls podName:83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:09.314477015 +0000 UTC m=+43.628181901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls") pod "image-registry-756d85f79-ct589" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2") : secret "image-registry-tls" not found Jul 15 16:59:05.314533 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.314517 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:09.314506844 +0000 UTC m=+43.628211722 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : configmap references non-existent config key: service-ca.crt Jul 15 16:59:05.314825 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.314541 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:09.31452898 +0000 UTC m=+43.628233850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : secret "router-metrics-certs-default" not found Jul 15 16:59:05.314825 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.314485 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:05.314825 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.314634 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls podName:8c9266b8-1422-4e7a-9736-89316363fc85 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:09.314619904 +0000 UTC m=+43.628324777 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wd9bp" (UID: "8c9266b8-1422-4e7a-9736-89316363fc85") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:05.415061 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.415024 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:05.415650 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.415159 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:05.415650 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.415218 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 16:59:05.415650 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.415274 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:59:05.415650 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.415296 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls podName:b504a6a8-6759-41b5-960f-83ffe042f01a nodeName:}" failed. No retries permitted until 2026-07-15 16:59:06.415277183 +0000 UTC m=+40.728982067 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls") pod "cluster-samples-operator-65db858894-wp8hf" (UID: "b504a6a8-6759-41b5-960f-83ffe042f01a") : secret "samples-operator-tls" not found Jul 15 16:59:05.415650 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:05.415318 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls podName:30da06ef-d2dd-4cdf-bcfa-02bf28378483 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:09.415302942 +0000 UTC m=+43.729007812 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls") pod "dns-default-jht7b" (UID: "30da06ef-d2dd-4cdf-bcfa-02bf28378483") : secret "dns-default-metrics-tls" not found Jul 15 16:59:05.468751 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.468711 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:05.621185 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.621150 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:05.850856 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.850820 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:05.927199 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:05.927146 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:06.425796 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:06.425760 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:06.426373 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:06.425953 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 16:59:06.426373 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:06.426037 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls podName:b504a6a8-6759-41b5-960f-83ffe042f01a nodeName:}" failed. No retries permitted until 2026-07-15 16:59:08.426015171 +0000 UTC m=+42.739720057 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls") pod "cluster-samples-operator-65db858894-wp8hf" (UID: "b504a6a8-6759-41b5-960f-83ffe042f01a") : secret "samples-operator-tls" not found Jul 15 16:59:06.815341 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:06.815153 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb"] Jul 15 16:59:06.818642 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:06.818594 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a3ef658_21e3_409d_973a_ef9afbbb3b46.slice/crio-b1895a9cc71b909263412ee8eb48f23410793a0173319015ca767526420c728d WatchSource:0}: Error finding container b1895a9cc71b909263412ee8eb48f23410793a0173319015ca767526420c728d: Status 404 returned error can't find the container with id b1895a9cc71b909263412ee8eb48f23410793a0173319015ca767526420c728d Jul 15 16:59:06.827543 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:06.827515 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd"] Jul 15 16:59:06.827678 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:06.827577 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-bb8gz"] Jul 15 16:59:06.837638 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:06.833335 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f3aa3b2_8da9_47a5_a23e_72974a650bac.slice/crio-0f35c9b3aee7cf3845d4c843c923732a14bf35059f74cae6565200ae1770721b WatchSource:0}: Error finding container 0f35c9b3aee7cf3845d4c843c923732a14bf35059f74cae6565200ae1770721b: Status 404 returned error can't find the container with id 0f35c9b3aee7cf3845d4c843c923732a14bf35059f74cae6565200ae1770721b Jul 15 16:59:07.517717 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.516852 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v78pr" event={"ID":"3ee02540-b301-416f-9ac2-41c5170668c4","Type":"ContainerStarted","Data":"e779c3dd509c75d9c9e7712f63cd53cfb3668c49d90f43529a1e65a7f67a4e5f"} Jul 15 16:59:07.525633 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.522877 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-bb8gz" event={"ID":"4f3aa3b2-8da9-47a5-a23e-72974a650bac","Type":"ContainerStarted","Data":"0f35c9b3aee7cf3845d4c843c923732a14bf35059f74cae6565200ae1770721b"} Jul 15 16:59:07.526914 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.526891 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" event={"ID":"5d65117a-a7e7-4feb-9a0f-f05010d0c737","Type":"ContainerStarted","Data":"e2c468b1945e047aff38e90752f60ee3dbfb773d48509304ae87bdd73931b285"} Jul 15 16:59:07.529212 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.528837 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" event={"ID":"1c975720-099d-446b-bcf7-cb34eb853f8a","Type":"ContainerStarted","Data":"39026a8175e4f8763475f874b5c5249e58abf025ed05b07df215b64dc2984528"} Jul 15 16:59:07.530848 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.530796 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" event={"ID":"63bf6178-a18e-4b9c-81bb-98a008e87fc8","Type":"ContainerStarted","Data":"62beab362460c058e54969e85c776ef7d2999a4641b4925fd87b0d44f35a4a4e"} Jul 15 16:59:07.532173 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.532150 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd" event={"ID":"596e6e61-c5fe-449a-833e-861d4588e9fb","Type":"ContainerStarted","Data":"c899c8679bf1ffcde7d52ce907d091b38f4a835ec6e49509322a121e05f9ba22"} Jul 15 16:59:07.533764 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.533741 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" event={"ID":"8a3ef658-21e3-409d-973a-ef9afbbb3b46","Type":"ContainerStarted","Data":"b1895a9cc71b909263412ee8eb48f23410793a0173319015ca767526420c728d"} Jul 15 16:59:07.541631 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.538827 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v78pr" podStartSLOduration=2.5630860330000003 podStartE2EDuration="6.538809008s" totalImagesPullingTime="3.975722975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:07.537925297 +0000 UTC m=+41.851630190" watchObservedRunningTime="2026-07-15 16:59:07.538809008 +0000 UTC m=+41.852513914" Jul 15 16:59:07.561193 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.560764 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" podStartSLOduration=9.583205887 podStartE2EDuration="13.560743006s" totalImagesPullingTime="3.977537119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:07.559344091 +0000 UTC m=+41.873048985" watchObservedRunningTime="2026-07-15 16:59:07.560743006 +0000 UTC m=+41.874447901" Jul 15 16:59:07.614321 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.614261 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" podStartSLOduration=9.63189052 podStartE2EDuration="13.614237567s" totalImagesPullingTime="3.982347047s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:07.579347341 +0000 UTC m=+41.893052234" watchObservedRunningTime="2026-07-15 16:59:07.614237567 +0000 UTC m=+41.927942460" Jul 15 16:59:07.735525 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.735286 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:07.836193 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.836140 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:07.951360 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:07.951310 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:08.053042 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.052884 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:08.154932 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.154721 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:08.387396 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.387362 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:08.448587 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.448504 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:08.448767 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:08.448717 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 16:59:08.448825 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:08.448802 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls podName:b504a6a8-6759-41b5-960f-83ffe042f01a nodeName:}" failed. No retries permitted until 2026-07-15 16:59:12.448778365 +0000 UTC m=+46.762483256 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls") pod "cluster-samples-operator-65db858894-wp8hf" (UID: "b504a6a8-6759-41b5-960f-83ffe042f01a") : secret "samples-operator-tls" not found Jul 15 16:59:08.463287 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.463237 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:08.697940 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.697908 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:08.797056 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.796905 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mxzc4" podStartSLOduration=12.280492062 podStartE2EDuration="42.796880439s" totalImagesPullingTime="30.516388377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:07.617399151 +0000 UTC m=+41.931104039" watchObservedRunningTime="2026-07-15 16:59:08.796880439 +0000 UTC m=+43.110585332" Jul 15 16:59:08.797539 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.797504 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4nrmg"] Jul 15 16:59:08.802905 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.802878 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.805759 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.805737 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2tcfp\"" Jul 15 16:59:08.805759 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.805736 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 16:59:08.805966 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.805737 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 16:59:08.818350 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.818319 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4nrmg"] Jul 15 16:59:08.852527 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.852491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.852728 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.852547 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.852728 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.852572 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlwk6\" (UniqueName: \"kubernetes.io/projected/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-api-access-vlwk6\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.852728 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.852688 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f96fcf3d-4d29-4a71-a191-cb0b609a1602-data-volume\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.852882 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.852748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f96fcf3d-4d29-4a71-a191-cb0b609a1602-crio-socket\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.954169 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.954128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f96fcf3d-4d29-4a71-a191-cb0b609a1602-data-volume\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.954360 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.954191 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f96fcf3d-4d29-4a71-a191-cb0b609a1602-crio-socket\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.954427 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.954362 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.954427 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.954398 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f96fcf3d-4d29-4a71-a191-cb0b609a1602-crio-socket\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.954529 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.954424 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.954529 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.954454 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwk6\" (UniqueName: \"kubernetes.io/projected/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-api-access-vlwk6\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.954529 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.954460 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f96fcf3d-4d29-4a71-a191-cb0b609a1602-data-volume\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:08.954529 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:08.954499 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:59:08.954751 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:08.954541 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:59:08.954751 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:08.954559 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:09.454542992 +0000 UTC m=+43.768247862 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : secret "insights-runtime-extractor-tls" not found Jul 15 16:59:08.954751 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:08.954627 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:09.454589279 +0000 UTC m=+43.768294149 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : configmap "kube-rbac-proxy" not found Jul 15 16:59:08.966953 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:08.966922 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlwk6\" (UniqueName: \"kubernetes.io/projected/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-api-access-vlwk6\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:09.173910 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:09.173880 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:09.247421 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:09.247380 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:09.357353 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:09.357316 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:09.357526 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:09.357383 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:09.357526 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:09.357439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:09.357526 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.357441 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 16:59:09.357526 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.357466 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-756d85f79-ct589: secret "image-registry-tls" not found Jul 15 16:59:09.357526 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:09.357472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:09.357794 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.357543 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:17.357521814 +0000 UTC m=+51.671226690 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : configmap references non-existent config key: service-ca.crt Jul 15 16:59:09.357794 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.357570 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls podName:83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:17.357561212 +0000 UTC m=+51.671266088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls") pod "image-registry-756d85f79-ct589" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2") : secret "image-registry-tls" not found Jul 15 16:59:09.357794 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.357574 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:09.357794 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.357591 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:59:09.357794 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.357648 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls podName:8c9266b8-1422-4e7a-9736-89316363fc85 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:17.357631305 +0000 UTC m=+51.671336178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wd9bp" (UID: "8c9266b8-1422-4e7a-9736-89316363fc85") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:09.357794 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.357671 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:17.357661359 +0000 UTC m=+51.671366301 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : secret "router-metrics-certs-default" not found Jul 15 16:59:09.458783 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:09.458694 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:09.458944 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:09.458835 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:09.458944 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.458874 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:59:09.458944 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:09.458896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:09.458944 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.458909 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:59:09.459162 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.458962 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls podName:30da06ef-d2dd-4cdf-bcfa-02bf28378483 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:17.458941903 +0000 UTC m=+51.772646772 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls") pod "dns-default-jht7b" (UID: "30da06ef-d2dd-4cdf-bcfa-02bf28378483") : secret "dns-default-metrics-tls" not found Jul 15 16:59:09.459162 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.458991 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:59:09.459162 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.459002 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:10.458978768 +0000 UTC m=+44.772683650 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : secret "insights-runtime-extractor-tls" not found Jul 15 16:59:09.459162 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:09.459040 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:10.459029277 +0000 UTC m=+44.772734353 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : configmap "kube-rbac-proxy" not found Jul 15 16:59:10.087211 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.087175 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t8xnk_2d7dc39f-d2fc-42e9-9338-decd2e825747/dns-node-resolver/0.log" Jul 15 16:59:10.263379 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.263357 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-cx87p"] Jul 15 16:59:10.293415 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.293176 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-cx87p"] Jul 15 16:59:10.293415 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.293271 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.297042 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.296834 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-nh7vx\"" Jul 15 16:59:10.297842 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.297538 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 15 16:59:10.297842 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.297560 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 15 16:59:10.297842 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.297824 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 15 16:59:10.298006 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.297842 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 15 16:59:10.321000 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.320940 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:10.367296 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.367268 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8ae77eb-6703-4fc7-995d-a68a2785416c-signing-cabundle\") pod \"service-ca-79f68b9f4f-cx87p\" (UID: \"b8ae77eb-6703-4fc7-995d-a68a2785416c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.367402 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.367368 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8ae77eb-6703-4fc7-995d-a68a2785416c-signing-key\") pod \"service-ca-79f68b9f4f-cx87p\" (UID: \"b8ae77eb-6703-4fc7-995d-a68a2785416c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.367442 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.367401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z27l8\" (UniqueName: \"kubernetes.io/projected/b8ae77eb-6703-4fc7-995d-a68a2785416c-kube-api-access-z27l8\") pod \"service-ca-79f68b9f4f-cx87p\" (UID: \"b8ae77eb-6703-4fc7-995d-a68a2785416c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.423922 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.423888 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:10.468146 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.468064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:10.468146 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.468101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8ae77eb-6703-4fc7-995d-a68a2785416c-signing-key\") pod \"service-ca-79f68b9f4f-cx87p\" (UID: \"b8ae77eb-6703-4fc7-995d-a68a2785416c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.468322 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:10.468221 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:59:10.468322 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.468241 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z27l8\" (UniqueName: \"kubernetes.io/projected/b8ae77eb-6703-4fc7-995d-a68a2785416c-kube-api-access-z27l8\") pod \"service-ca-79f68b9f4f-cx87p\" (UID: \"b8ae77eb-6703-4fc7-995d-a68a2785416c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.468322 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.468283 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:10.468438 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:10.468383 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:12.468364265 +0000 UTC m=+46.782069141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : secret "insights-runtime-extractor-tls" not found Jul 15 16:59:10.468438 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:10.468398 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:59:10.468526 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:10.468469 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:12.468451924 +0000 UTC m=+46.782156819 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : configmap "kube-rbac-proxy" not found Jul 15 16:59:10.468574 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.468532 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8ae77eb-6703-4fc7-995d-a68a2785416c-signing-cabundle\") pod \"service-ca-79f68b9f4f-cx87p\" (UID: \"b8ae77eb-6703-4fc7-995d-a68a2785416c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.469155 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.469138 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b8ae77eb-6703-4fc7-995d-a68a2785416c-signing-cabundle\") pod \"service-ca-79f68b9f4f-cx87p\" (UID: \"b8ae77eb-6703-4fc7-995d-a68a2785416c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.470692 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.470673 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b8ae77eb-6703-4fc7-995d-a68a2785416c-signing-key\") pod \"service-ca-79f68b9f4f-cx87p\" (UID: \"b8ae77eb-6703-4fc7-995d-a68a2785416c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.478202 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.478182 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z27l8\" (UniqueName: \"kubernetes.io/projected/b8ae77eb-6703-4fc7-995d-a68a2785416c-kube-api-access-z27l8\") pod \"service-ca-79f68b9f4f-cx87p\" (UID: \"b8ae77eb-6703-4fc7-995d-a68a2785416c\") " pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.486488 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.486472 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pxcv_25eeab2a-5a16-4d82-a085-22f22a60c078/node-ca/0.log" Jul 15 16:59:10.545233 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.545192 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd" event={"ID":"596e6e61-c5fe-449a-833e-861d4588e9fb","Type":"ContainerStarted","Data":"5191223f0a7f7b3dd49aec1a3106f4192ddff17df16b52899a60cc1f3a719d78"} Jul 15 16:59:10.546846 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.546813 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" event={"ID":"8a3ef658-21e3-409d-973a-ef9afbbb3b46","Type":"ContainerStarted","Data":"4472ce4ac4b593f1f42b9e08996b8cf544aa827597b7e4a9c264a2bdc4e40f18"} Jul 15 16:59:10.548379 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.548357 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/0.log" Jul 15 16:59:10.548521 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.548396 2566 generic.go:356] "Generic (PLEG): container finished" podID="4f3aa3b2-8da9-47a5-a23e-72974a650bac" containerID="3df3dd99e6d393e9569634ad301e6a05bc6806f4662b034d66b4fa90efa7bde5" exitCode=255 Jul 15 16:59:10.548521 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.548433 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-bb8gz" event={"ID":"4f3aa3b2-8da9-47a5-a23e-72974a650bac","Type":"ContainerDied","Data":"3df3dd99e6d393e9569634ad301e6a05bc6806f4662b034d66b4fa90efa7bde5"} Jul 15 16:59:10.548771 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.548754 2566 scope.go:117] "RemoveContainer" containerID="3df3dd99e6d393e9569634ad301e6a05bc6806f4662b034d66b4fa90efa7bde5" Jul 15 16:59:10.563372 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.563333 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-gldzd" podStartSLOduration=3.268189376 podStartE2EDuration="6.563293046s" totalImagesPullingTime="3.29510367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:10.562897379 +0000 UTC m=+44.876602273" watchObservedRunningTime="2026-07-15 16:59:10.563293046 +0000 UTC m=+44.876997989" Jul 15 16:59:10.584441 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.584338 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" podStartSLOduration=3.245404583 podStartE2EDuration="6.584317987s" totalImagesPullingTime="3.338913404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:10.5838613 +0000 UTC m=+44.897566207" watchObservedRunningTime="2026-07-15 16:59:10.584317987 +0000 UTC m=+44.898022880" Jul 15 16:59:10.605355 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.605323 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" Jul 15 16:59:10.738341 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.738306 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:10.770436 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.770403 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-cx87p"] Jul 15 16:59:10.773886 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:10.773845 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ae77eb_6703_4fc7_995d_a68a2785416c.slice/crio-f26693c61bdf3f0790e0ea7af866ed94440048a9374939cd83a97ddf75c9c8a4 WatchSource:0}: Error finding container f26693c61bdf3f0790e0ea7af866ed94440048a9374939cd83a97ddf75c9c8a4: Status 404 returned error can't find the container with id f26693c61bdf3f0790e0ea7af866ed94440048a9374939cd83a97ddf75c9c8a4 Jul 15 16:59:10.836089 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:10.836052 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:11.093321 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.093284 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v78pr_3ee02540-b301-416f-9ac2-41c5170668c4/serve-healthcheck-canary/0.log" Jul 15 16:59:11.205218 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.205182 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:11.307913 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.307877 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:11.408598 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.408516 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:11.553784 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.553758 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 16:59:11.554197 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.554180 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/0.log" Jul 15 16:59:11.554258 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.554221 2566 generic.go:356] "Generic (PLEG): container finished" podID="4f3aa3b2-8da9-47a5-a23e-72974a650bac" containerID="8c2d00174ab6193dfbe66ef3f0858b192c1d2d72ac89b70acaf9a02f69fcca16" exitCode=255 Jul 15 16:59:11.554323 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.554301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-bb8gz" event={"ID":"4f3aa3b2-8da9-47a5-a23e-72974a650bac","Type":"ContainerDied","Data":"8c2d00174ab6193dfbe66ef3f0858b192c1d2d72ac89b70acaf9a02f69fcca16"} Jul 15 16:59:11.554392 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.554377 2566 scope.go:117] "RemoveContainer" containerID="3df3dd99e6d393e9569634ad301e6a05bc6806f4662b034d66b4fa90efa7bde5" Jul 15 16:59:11.554997 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.554572 2566 scope.go:117] "RemoveContainer" containerID="8c2d00174ab6193dfbe66ef3f0858b192c1d2d72ac89b70acaf9a02f69fcca16" Jul 15 16:59:11.554997 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:11.554806 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-bb8gz_openshift-console-operator(4f3aa3b2-8da9-47a5-a23e-72974a650bac)\"" pod="openshift-console-operator/console-operator-7866497494-bb8gz" podUID="4f3aa3b2-8da9-47a5-a23e-72974a650bac" Jul 15 16:59:11.555994 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.555973 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" event={"ID":"b8ae77eb-6703-4fc7-995d-a68a2785416c","Type":"ContainerStarted","Data":"8a32419251c05d7b59710b362a8d38e1e96a09e926d29bd2b740f9a94e54ea5f"} Jul 15 16:59:11.556087 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.556014 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" event={"ID":"b8ae77eb-6703-4fc7-995d-a68a2785416c","Type":"ContainerStarted","Data":"f26693c61bdf3f0790e0ea7af866ed94440048a9374939cd83a97ddf75c9c8a4"} Jul 15 16:59:11.577394 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.577366 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:11.630908 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.630858 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-cx87p" podStartSLOduration=1.630841755 podStartE2EDuration="1.630841755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:11.630267261 +0000 UTC m=+45.943972154" watchObservedRunningTime="2026-07-15 16:59:11.630841755 +0000 UTC m=+45.944546648" Jul 15 16:59:11.681913 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.681818 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:11.927691 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:11.927658 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:12.101559 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:12.101526 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:12.489956 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:12.489857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:12.489956 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:12.489914 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:12.490189 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:12.489967 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:12.490189 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:12.489975 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 15 16:59:12.490189 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:12.490050 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls podName:b504a6a8-6759-41b5-960f-83ffe042f01a nodeName:}" failed. No retries permitted until 2026-07-15 16:59:20.490033058 +0000 UTC m=+54.803737928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls") pod "cluster-samples-operator-65db858894-wp8hf" (UID: "b504a6a8-6759-41b5-960f-83ffe042f01a") : secret "samples-operator-tls" not found Jul 15 16:59:12.490189 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:12.490068 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:59:12.490189 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:12.490076 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:59:12.490189 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:12.490126 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:16.490110634 +0000 UTC m=+50.803815503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : configmap "kube-rbac-proxy" not found Jul 15 16:59:12.490189 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:12.490143 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:16.490134452 +0000 UTC m=+50.803839325 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : secret "insights-runtime-extractor-tls" not found Jul 15 16:59:12.562781 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:12.562745 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 16:59:12.563568 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:12.563546 2566 scope.go:117] "RemoveContainer" containerID="8c2d00174ab6193dfbe66ef3f0858b192c1d2d72ac89b70acaf9a02f69fcca16" Jul 15 16:59:12.564469 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:12.564440 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-bb8gz_openshift-console-operator(4f3aa3b2-8da9-47a5-a23e-72974a650bac)\"" pod="openshift-console-operator/console-operator-7866497494-bb8gz" podUID="4f3aa3b2-8da9-47a5-a23e-72974a650bac" Jul 15 16:59:12.716123 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:12.716083 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:12.807075 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:12.806985 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:12.927070 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:12.927036 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:13.068492 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:13.068405 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:13.195898 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:13.195866 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:13.553698 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:13.553654 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:13.939157 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:13.939126 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:14.923808 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:14.923780 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:14.926545 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:14.926528 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:14.926655 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:14.926567 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:14.926931 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:14.926914 2566 scope.go:117] "RemoveContainer" containerID="8c2d00174ab6193dfbe66ef3f0858b192c1d2d72ac89b70acaf9a02f69fcca16" Jul 15 16:59:14.927152 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:14.927130 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-bb8gz_openshift-console-operator(4f3aa3b2-8da9-47a5-a23e-72974a650bac)\"" pod="openshift-console-operator/console-operator-7866497494-bb8gz" podUID="4f3aa3b2-8da9-47a5-a23e-72974a650bac" Jul 15 16:59:15.045412 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:15.045375 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:15.351799 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:15.351752 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:15.736367 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:15.736252 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:16.215896 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:16.215851 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:16.350275 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:16.350241 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:16.526075 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:16.525982 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:16.526075 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:16.526041 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:16.526298 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:16.526125 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:59:16.526298 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:16.526166 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:59:16.526298 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:16.526192 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:24.52617708 +0000 UTC m=+58.839881949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : configmap "kube-rbac-proxy" not found Jul 15 16:59:16.526298 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:16.526241 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls podName:f96fcf3d-4d29-4a71-a191-cb0b609a1602 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:24.526222987 +0000 UTC m=+58.839927860 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4nrmg" (UID: "f96fcf3d-4d29-4a71-a191-cb0b609a1602") : secret "insights-runtime-extractor-tls" not found Jul 15 16:59:16.937235 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:16.937192 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:17.087355 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.087320 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:17.398934 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.398906 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:17.434455 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.434423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:17.434655 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.434459 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:17.434655 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:17.434574 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:17.434655 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:17.434656 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls podName:8c9266b8-1422-4e7a-9736-89316363fc85 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:33.43463828 +0000 UTC m=+67.748343150 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-wd9bp" (UID: "8c9266b8-1422-4e7a-9736-89316363fc85") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:59:17.434837 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.434725 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:17.434837 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.434757 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:17.434930 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:17.434842 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle podName:9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:59:33.43483382 +0000 UTC m=+67.748538690 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle") pod "router-default-7d99b5d79b-cfd7r" (UID: "9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1") : configmap references non-existent config key: service-ca.crt Jul 15 16:59:17.437058 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.437038 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-metrics-certs\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:17.437156 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.437133 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") pod \"image-registry-756d85f79-ct589\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:17.479262 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.479226 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:17.536050 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.536009 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:17.538474 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.538443 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/30da06ef-d2dd-4cdf-bcfa-02bf28378483-metrics-tls\") pod \"dns-default-jht7b\" (UID: \"30da06ef-d2dd-4cdf-bcfa-02bf28378483\") " pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:17.647666 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.647619 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-756d85f79-ct589"] Jul 15 16:59:17.651204 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:17.651150 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83dd4c57_2b5b_4da1_8d69_a6a1e8652fb2.slice/crio-69ce7778f5546ff9153da063de62560275ba44049fa67ea83ad6aa4fa7cbe930 WatchSource:0}: Error finding container 69ce7778f5546ff9153da063de62560275ba44049fa67ea83ad6aa4fa7cbe930: Status 404 returned error can't find the container with id 69ce7778f5546ff9153da063de62560275ba44049fa67ea83ad6aa4fa7cbe930 Jul 15 16:59:17.757014 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.756983 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:17.824030 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.823999 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:17.976138 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.976065 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:17.983223 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:17.983198 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jht7b"] Jul 15 16:59:17.986574 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:17.986537 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30da06ef_d2dd_4cdf_bcfa_02bf28378483.slice/crio-e189e94d92e95b6bd24f660f93f86f67169bc8fd55817d789804f79ba6be58f4 WatchSource:0}: Error finding container e189e94d92e95b6bd24f660f93f86f67169bc8fd55817d789804f79ba6be58f4: Status 404 returned error can't find the container with id e189e94d92e95b6bd24f660f93f86f67169bc8fd55817d789804f79ba6be58f4 Jul 15 16:59:18.092579 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:18.092542 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:18.201947 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:18.201868 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:18.534525 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:18.534449 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:18.585969 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:18.585929 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jht7b" event={"ID":"30da06ef-d2dd-4cdf-bcfa-02bf28378483","Type":"ContainerStarted","Data":"e189e94d92e95b6bd24f660f93f86f67169bc8fd55817d789804f79ba6be58f4"} Jul 15 16:59:18.588080 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:18.588042 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-756d85f79-ct589" event={"ID":"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2","Type":"ContainerStarted","Data":"b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6"} Jul 15 16:59:18.588230 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:18.588096 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-756d85f79-ct589" event={"ID":"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2","Type":"ContainerStarted","Data":"69ce7778f5546ff9153da063de62560275ba44049fa67ea83ad6aa4fa7cbe930"} Jul 15 16:59:18.588230 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:18.588134 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:18.945003 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:18.944967 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:19.063882 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:19.063817 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:19.325096 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:19.325017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:19.509196 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:19.509160 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:19.796589 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:19.796553 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:20.563559 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.563528 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:20.566250 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.566225 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b504a6a8-6759-41b5-960f-83ffe042f01a-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wp8hf\" (UID: \"b504a6a8-6759-41b5-960f-83ffe042f01a\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:20.598416 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.598375 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jht7b" event={"ID":"30da06ef-d2dd-4cdf-bcfa-02bf28378483","Type":"ContainerStarted","Data":"1a4f5b59d4dab55179394322b4925041b21eec90969b8d3e6cbe798df39473ed"} Jul 15 16:59:20.598587 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.598446 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jht7b" event={"ID":"30da06ef-d2dd-4cdf-bcfa-02bf28378483","Type":"ContainerStarted","Data":"d2357324dc3b5d62b53eaf8f2fbefa1f1f797aadf2f37c6d95ea0b8171780a94"} Jul 15 16:59:20.598587 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.598475 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:20.619512 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.619423 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jht7b" podStartSLOduration=17.256901833 podStartE2EDuration="19.619407904s" totalImagesPullingTime="2.362506071s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:20.61776899 +0000 UTC m=+54.931473936" watchObservedRunningTime="2026-07-15 16:59:20.619407904 +0000 UTC m=+54.933112795" Jul 15 16:59:20.619872 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.619846 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-756d85f79-ct589" podStartSLOduration=54.619837638999996 podStartE2EDuration="54.619837639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:18.622160912 +0000 UTC m=+52.935865806" watchObservedRunningTime="2026-07-15 16:59:20.619837639 +0000 UTC m=+54.933542532" Jul 15 16:59:20.621998 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.621976 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" Jul 15 16:59:20.707494 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.707463 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:20.754678 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:20.754645 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf"] Jul 15 16:59:21.159358 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:21.159323 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:21.259411 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:21.259372 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:21.538952 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:21.538830 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:21.604738 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:21.604696 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" event={"ID":"b504a6a8-6759-41b5-960f-83ffe042f01a","Type":"ContainerStarted","Data":"c8b439f619013b869a4fac0db7e4278999c0023996b1f03db70c98ecf598d430"} Jul 15 16:59:22.155534 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:22.155498 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:22.611162 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:22.611122 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" event={"ID":"b504a6a8-6759-41b5-960f-83ffe042f01a","Type":"ContainerStarted","Data":"ad82d442c6e6eef05633d7f9b9e2a50d61f1047185b0ff4a052f89e10395b401"} Jul 15 16:59:22.611574 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:22.611184 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" event={"ID":"b504a6a8-6759-41b5-960f-83ffe042f01a","Type":"ContainerStarted","Data":"2e5f862633a4df2d3a957dec5b2d89735fed751e2bc76b14875b75a8b3285a65"} Jul 15 16:59:22.638614 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:22.638558 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wp8hf" podStartSLOduration=16.975247744 podStartE2EDuration="18.638542087s" totalImagesPullingTime="1.663294343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:22.637777189 +0000 UTC m=+56.951482084" watchObservedRunningTime="2026-07-15 16:59:22.638542087 +0000 UTC m=+56.952246991" Jul 15 16:59:22.744878 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:22.744805 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:22.927364 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:22.927334 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:23.129389 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:23.129350 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:23.402133 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:23.402030 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" limit=1024 Jul 15 16:59:24.600396 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:24.600348 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:24.600906 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:24.600406 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:24.600906 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:24.600890 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f96fcf3d-4d29-4a71-a191-cb0b609a1602-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:24.602879 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:24.602853 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f96fcf3d-4d29-4a71-a191-cb0b609a1602-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4nrmg\" (UID: \"f96fcf3d-4d29-4a71-a191-cb0b609a1602\") " pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:24.714326 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:24.714283 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4nrmg" Jul 15 16:59:24.858780 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:24.858750 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4nrmg"] Jul 15 16:59:24.861295 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:24.861266 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96fcf3d_4d29_4a71_a191_cb0b609a1602.slice/crio-45a176facf4c5ea3f77c783e80afd698da525a5a75ad04a3b5c6d9ab91f0b9f8 WatchSource:0}: Error finding container 45a176facf4c5ea3f77c783e80afd698da525a5a75ad04a3b5c6d9ab91f0b9f8: Status 404 returned error can't find the container with id 45a176facf4c5ea3f77c783e80afd698da525a5a75ad04a3b5c6d9ab91f0b9f8 Jul 15 16:59:25.623940 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:25.623839 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4nrmg" event={"ID":"f96fcf3d-4d29-4a71-a191-cb0b609a1602","Type":"ContainerStarted","Data":"45a176facf4c5ea3f77c783e80afd698da525a5a75ad04a3b5c6d9ab91f0b9f8"} Jul 15 16:59:26.493502 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:26.493470 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-wb5hg" Jul 15 16:59:29.182355 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:29.182325 2566 scope.go:117] "RemoveContainer" containerID="8c2d00174ab6193dfbe66ef3f0858b192c1d2d72ac89b70acaf9a02f69fcca16" Jul 15 16:59:30.608149 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:30.608113 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jht7b" Jul 15 16:59:30.644283 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:30.644253 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 16:59:30.644539 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:30.644411 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-bb8gz" event={"ID":"4f3aa3b2-8da9-47a5-a23e-72974a650bac","Type":"ContainerStarted","Data":"f5910a49536ebeba0e62330ffc6bb14bd6d97ef15106d4c4d8530639aa76faa3"} Jul 15 16:59:30.644849 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:30.644825 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:30.648050 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:30.648022 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4nrmg" event={"ID":"f96fcf3d-4d29-4a71-a191-cb0b609a1602","Type":"ContainerStarted","Data":"962158254f71a65629105f71a22d53ce47c076708c32c1d1f7bb60e669e51844"} Jul 15 16:59:30.673276 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:30.673218 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-bb8gz" podStartSLOduration=23.356068111 podStartE2EDuration="26.673197619s" totalImagesPullingTime="3.317129508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:30.672624888 +0000 UTC m=+64.986329775" watchObservedRunningTime="2026-07-15 16:59:30.673197619 +0000 UTC m=+64.986902544" Jul 15 16:59:30.680138 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:30.680073 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-bb8gz" Jul 15 16:59:31.872051 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.871997 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:59:31.875683 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.875653 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 16:59:31.884887 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.884854 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1d2712c7-d910-4a49-900d-ae83e771b3c1-metrics-certs\") pod \"network-metrics-daemon-w7n7w\" (UID: \"1d2712c7-d910-4a49-900d-ae83e771b3c1\") " pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:59:31.897426 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.897399 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tns8n\"" Jul 15 16:59:31.904837 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.904806 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w7n7w" Jul 15 16:59:31.973522 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.973478 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:59:31.973688 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.973555 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:59:31.976825 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.976796 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 16:59:31.977236 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.977213 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 16:59:31.987036 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.987002 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7c43931a-91df-49ea-bea3-10e907fe1b5c-original-pull-secret\") pod \"global-pull-secret-syncer-d48rf\" (UID: \"7c43931a-91df-49ea-bea3-10e907fe1b5c\") " pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:59:31.987726 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.987706 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 16:59:31.998373 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:31.998344 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbnxw\" (UniqueName: \"kubernetes.io/projected/fc473a95-bf8d-46ac-b047-8be9d1c1efc3-kube-api-access-zbnxw\") pod \"network-check-target-m4wq5\" (UID: \"fc473a95-bf8d-46ac-b047-8be9d1c1efc3\") " pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:59:32.200539 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:32.200446 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d48rf" Jul 15 16:59:32.295390 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:32.295358 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f7t54\"" Jul 15 16:59:32.303233 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:32.303200 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:59:33.485828 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.485787 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:33.486389 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.485888 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:33.486716 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.486696 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1-service-ca-bundle\") pod \"router-default-7d99b5d79b-cfd7r\" (UID: \"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1\") " pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:33.489020 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.488991 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/8c9266b8-1422-4e7a-9736-89316363fc85-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-wd9bp\" (UID: \"8c9266b8-1422-4e7a-9736-89316363fc85\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:33.628215 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.628190 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d48rf"] Jul 15 16:59:33.691658 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.691624 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-4ghtm\"" Jul 15 16:59:33.699395 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.699374 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" Jul 15 16:59:33.716838 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.716779 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-xvxm7\"" Jul 15 16:59:33.723492 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.723463 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:33.791864 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:33.791832 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c43931a_91df_49ea_bea3_10e907fe1b5c.slice/crio-5912afcec4a2b64707db99d3282a8d5450efd481e6f2f6ae98e0076796113a12 WatchSource:0}: Error finding container 5912afcec4a2b64707db99d3282a8d5450efd481e6f2f6ae98e0076796113a12: Status 404 returned error can't find the container with id 5912afcec4a2b64707db99d3282a8d5450efd481e6f2f6ae98e0076796113a12 Jul 15 16:59:33.860301 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.860267 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w7n7w"] Jul 15 16:59:33.864555 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.864214 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m4wq5"] Jul 15 16:59:33.865058 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:33.865035 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2712c7_d910_4a49_900d_ae83e771b3c1.slice/crio-dad87e74b87a2b7d31482ebe2d73f31eddd5e3601d510f54cecc2ce947f54a40 WatchSource:0}: Error finding container dad87e74b87a2b7d31482ebe2d73f31eddd5e3601d510f54cecc2ce947f54a40: Status 404 returned error can't find the container with id dad87e74b87a2b7d31482ebe2d73f31eddd5e3601d510f54cecc2ce947f54a40 Jul 15 16:59:33.972578 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.972556 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7d99b5d79b-cfd7r"] Jul 15 16:59:33.974870 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:33.974828 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9038fd2e_b72d_46fe_8ac6_94f3e0bae3a1.slice/crio-967241acc29030e6ab25ab346043d6cfb701ae3e8b6fada5b4208289f11b81d8 WatchSource:0}: Error finding container 967241acc29030e6ab25ab346043d6cfb701ae3e8b6fada5b4208289f11b81d8: Status 404 returned error can't find the container with id 967241acc29030e6ab25ab346043d6cfb701ae3e8b6fada5b4208289f11b81d8 Jul 15 16:59:33.983284 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:33.983261 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp"] Jul 15 16:59:33.985565 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:33.985543 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9266b8_1422_4e7a_9736_89316363fc85.slice/crio-270587966f4e0f47a89e15aead84486559af5003bbf1116cf4b771b4670d959a WatchSource:0}: Error finding container 270587966f4e0f47a89e15aead84486559af5003bbf1116cf4b771b4670d959a: Status 404 returned error can't find the container with id 270587966f4e0f47a89e15aead84486559af5003bbf1116cf4b771b4670d959a Jul 15 16:59:34.667596 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.667516 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m4wq5" event={"ID":"fc473a95-bf8d-46ac-b047-8be9d1c1efc3","Type":"ContainerStarted","Data":"089c9b6de101d17ab04dc665ae8358495420ee2b0fe2c9de1e6980e6b96feea1"} Jul 15 16:59:34.669706 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.669640 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" event={"ID":"8c9266b8-1422-4e7a-9736-89316363fc85","Type":"ContainerStarted","Data":"270587966f4e0f47a89e15aead84486559af5003bbf1116cf4b771b4670d959a"} Jul 15 16:59:34.673955 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.673921 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w7n7w" event={"ID":"1d2712c7-d910-4a49-900d-ae83e771b3c1","Type":"ContainerStarted","Data":"dad87e74b87a2b7d31482ebe2d73f31eddd5e3601d510f54cecc2ce947f54a40"} Jul 15 16:59:34.675972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.675939 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d48rf" event={"ID":"7c43931a-91df-49ea-bea3-10e907fe1b5c","Type":"ContainerStarted","Data":"5912afcec4a2b64707db99d3282a8d5450efd481e6f2f6ae98e0076796113a12"} Jul 15 16:59:34.679352 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.679326 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4nrmg" event={"ID":"f96fcf3d-4d29-4a71-a191-cb0b609a1602","Type":"ContainerStarted","Data":"3eefed27a653c52650787c626e4614588735410559cff4960dfff9df4bb792ab"} Jul 15 16:59:34.681067 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.681042 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" event={"ID":"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1","Type":"ContainerStarted","Data":"84c422190e3fb36037ab98ff1cbf61b23b255e942b970ff19f4ec0e54cf03961"} Jul 15 16:59:34.681191 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.681089 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" event={"ID":"9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1","Type":"ContainerStarted","Data":"967241acc29030e6ab25ab346043d6cfb701ae3e8b6fada5b4208289f11b81d8"} Jul 15 16:59:34.717196 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.717133 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" podStartSLOduration=40.717107711 podStartE2EDuration="40.717107711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:34.716016923 +0000 UTC m=+69.029721815" watchObservedRunningTime="2026-07-15 16:59:34.717107711 +0000 UTC m=+69.030812604" Jul 15 16:59:34.724252 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.724225 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:34.727492 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:34.727463 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:35.686512 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:35.686443 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:35.688846 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:35.688644 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7d99b5d79b-cfd7r" Jul 15 16:59:36.695635 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.695578 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w7n7w" event={"ID":"1d2712c7-d910-4a49-900d-ae83e771b3c1","Type":"ContainerStarted","Data":"3693cde9686aa7baa88678a678a7108e586a042fc36e7736886c30ed86763122"} Jul 15 16:59:36.696089 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.695662 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w7n7w" event={"ID":"1d2712c7-d910-4a49-900d-ae83e771b3c1","Type":"ContainerStarted","Data":"4760482bbf0d0a7c1231a3dfcba4094c50a96d4959746c34df9cb598c8ddf230"} Jul 15 16:59:36.886820 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.886060 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w7n7w" podStartSLOduration=69.170155711 podStartE2EDuration="1m10.886037109s" totalImagesPullingTime="1.715881398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:36.726212646 +0000 UTC m=+71.039917541" watchObservedRunningTime="2026-07-15 16:59:36.886037109 +0000 UTC m=+71.199742001" Jul 15 16:59:36.891692 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.891537 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-76t88"] Jul 15 16:59:36.923295 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.923248 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr"] Jul 15 16:59:36.923514 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.923393 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" Jul 15 16:59:36.927101 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.926337 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6fkr9\"" Jul 15 16:59:36.927101 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.926749 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 15 16:59:36.927101 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.926872 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 15 16:59:36.937351 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.937327 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c"] Jul 15 16:59:36.937478 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.937442 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:36.940807 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.940776 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 15 16:59:36.941158 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.941123 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 15 16:59:36.941373 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.941356 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 15 16:59:36.942395 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.942376 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 15 16:59:36.955461 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.955434 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g"] Jul 15 16:59:36.955593 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.955553 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" Jul 15 16:59:36.960645 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.960103 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-rq97l\"" Jul 15 16:59:36.960645 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.960374 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 15 16:59:36.977899 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.977768 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-76t88"] Jul 15 16:59:36.977899 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.977817 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr"] Jul 15 16:59:36.977899 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.977844 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c"] Jul 15 16:59:36.977899 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.977862 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g"] Jul 15 16:59:36.978178 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.977987 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:36.979746 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.979719 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-xd962"] Jul 15 16:59:36.983109 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.983089 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 15 16:59:36.983367 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.983348 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 15 16:59:36.983852 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.983668 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 15 16:59:36.983852 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.983756 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 15 16:59:36.994035 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.993999 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw"] Jul 15 16:59:36.994225 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.994144 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-xd962" Jul 15 16:59:36.997646 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.997457 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wf8ql\"" Jul 15 16:59:36.997646 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.997506 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 16:59:36.997646 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:36.997554 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 16:59:37.008516 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.008494 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5"] Jul 15 16:59:37.008721 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.008696 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw" Jul 15 16:59:37.012177 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.012161 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-nwhph\"" Jul 15 16:59:37.031016 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.030860 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw"] Jul 15 16:59:37.031016 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.030898 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5"] Jul 15 16:59:37.031016 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.030919 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-xd962"] Jul 15 16:59:37.031016 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.031006 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" Jul 15 16:59:37.033928 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.033904 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 16:59:37.033928 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.033924 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 16:59:37.034275 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.034257 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-2w68h\"" Jul 15 16:59:37.122395 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122351 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06f2e5cb-6674-46b8-919f-1b9b9b578683-nginx-conf\") pod \"networking-console-plugin-6cf856758c-76t88\" (UID: \"06f2e5cb-6674-46b8-919f-1b9b9b578683\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" Jul 15 16:59:37.122578 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.122578 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122451 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c7780bdf-d41c-4331-a92c-eb22aafb0e0c-klusterlet-config\") pod \"klusterlet-addon-workmgr-d66fd7f67-9jldr\" (UID: \"c7780bdf-d41c-4331-a92c-eb22aafb0e0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.122578 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122480 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtx8w\" (UniqueName: \"kubernetes.io/projected/445343c7-3076-481a-8cf3-b83e5f6d66e8-kube-api-access-xtx8w\") pod \"network-check-source-869dcfcdcd-px4mw\" (UID: \"445343c7-3076-481a-8cf3-b83e5f6d66e8\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw" Jul 15 16:59:37.122578 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7780bdf-d41c-4331-a92c-eb22aafb0e0c-tmp\") pod \"klusterlet-addon-workmgr-d66fd7f67-9jldr\" (UID: \"c7780bdf-d41c-4331-a92c-eb22aafb0e0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.122578 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bee772d4-84d4-43b8-8be9-26885da39d30-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-857b57b456-ns76c\" (UID: \"bee772d4-84d4-43b8-8be9-26885da39d30\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" Jul 15 16:59:37.122872 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122680 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.122872 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122714 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnknl\" (UniqueName: \"kubernetes.io/projected/cdeee4e8-f5bd-466e-94da-e7934bab2620-kube-api-access-bnknl\") pod \"downloads-68c9cbb497-xd962\" (UID: \"cdeee4e8-f5bd-466e-94da-e7934bab2620\") " pod="openshift-console/downloads-68c9cbb497-xd962" Jul 15 16:59:37.122872 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122753 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtsx9\" (UniqueName: \"kubernetes.io/projected/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-kube-api-access-qtsx9\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.122872 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122782 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7qx7\" (UniqueName: \"kubernetes.io/projected/c7780bdf-d41c-4331-a92c-eb22aafb0e0c-kube-api-access-m7qx7\") pod \"klusterlet-addon-workmgr-d66fd7f67-9jldr\" (UID: \"c7780bdf-d41c-4331-a92c-eb22aafb0e0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.122872 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-hub\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.122872 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.122872 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122863 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-ca\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.123211 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122897 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06f2e5cb-6674-46b8-919f-1b9b9b578683-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-76t88\" (UID: \"06f2e5cb-6674-46b8-919f-1b9b9b578683\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" Jul 15 16:59:37.123211 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.122932 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhtht\" (UniqueName: \"kubernetes.io/projected/bee772d4-84d4-43b8-8be9-26885da39d30-kube-api-access-qhtht\") pod \"managed-serviceaccount-addon-agent-857b57b456-ns76c\" (UID: \"bee772d4-84d4-43b8-8be9-26885da39d30\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" Jul 15 16:59:37.224041 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.223959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtsx9\" (UniqueName: \"kubernetes.io/projected/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-kube-api-access-qtsx9\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.224041 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224000 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7qx7\" (UniqueName: \"kubernetes.io/projected/c7780bdf-d41c-4331-a92c-eb22aafb0e0c-kube-api-access-m7qx7\") pod \"klusterlet-addon-workmgr-d66fd7f67-9jldr\" (UID: \"c7780bdf-d41c-4331-a92c-eb22aafb0e0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.224041 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-hub\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.224041 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224052 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-ca\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224070 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06f2e5cb-6674-46b8-919f-1b9b9b578683-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-76t88\" (UID: \"06f2e5cb-6674-46b8-919f-1b9b9b578683\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224097 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhtht\" (UniqueName: \"kubernetes.io/projected/bee772d4-84d4-43b8-8be9-26885da39d30-kube-api-access-qhtht\") pod \"managed-serviceaccount-addon-agent-857b57b456-ns76c\" (UID: \"bee772d4-84d4-43b8-8be9-26885da39d30\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mlh5\" (UniqueName: \"kubernetes.io/projected/055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe-kube-api-access-2mlh5\") pod \"migrator-7f59b54f6c-l9pf5\" (UID: \"055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224171 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06f2e5cb-6674-46b8-919f-1b9b9b578683-nginx-conf\") pod \"networking-console-plugin-6cf856758c-76t88\" (UID: \"06f2e5cb-6674-46b8-919f-1b9b9b578683\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224201 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224228 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c7780bdf-d41c-4331-a92c-eb22aafb0e0c-klusterlet-config\") pod \"klusterlet-addon-workmgr-d66fd7f67-9jldr\" (UID: \"c7780bdf-d41c-4331-a92c-eb22aafb0e0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtx8w\" (UniqueName: \"kubernetes.io/projected/445343c7-3076-481a-8cf3-b83e5f6d66e8-kube-api-access-xtx8w\") pod \"network-check-source-869dcfcdcd-px4mw\" (UID: \"445343c7-3076-481a-8cf3-b83e5f6d66e8\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7780bdf-d41c-4331-a92c-eb22aafb0e0c-tmp\") pod \"klusterlet-addon-workmgr-d66fd7f67-9jldr\" (UID: \"c7780bdf-d41c-4331-a92c-eb22aafb0e0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.224315 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224314 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bee772d4-84d4-43b8-8be9-26885da39d30-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-857b57b456-ns76c\" (UID: \"bee772d4-84d4-43b8-8be9-26885da39d30\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" Jul 15 16:59:37.225446 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.225446 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.224405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnknl\" (UniqueName: \"kubernetes.io/projected/cdeee4e8-f5bd-466e-94da-e7934bab2620-kube-api-access-bnknl\") pod \"downloads-68c9cbb497-xd962\" (UID: \"cdeee4e8-f5bd-466e-94da-e7934bab2620\") " pod="openshift-console/downloads-68c9cbb497-xd962" Jul 15 16:59:37.225446 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.225260 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.225446 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.225271 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/06f2e5cb-6674-46b8-919f-1b9b9b578683-nginx-conf\") pod \"networking-console-plugin-6cf856758c-76t88\" (UID: \"06f2e5cb-6674-46b8-919f-1b9b9b578683\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" Jul 15 16:59:37.225668 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.225622 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c7780bdf-d41c-4331-a92c-eb22aafb0e0c-tmp\") pod \"klusterlet-addon-workmgr-d66fd7f67-9jldr\" (UID: \"c7780bdf-d41c-4331-a92c-eb22aafb0e0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.227981 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.227960 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-hub\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.228581 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.228536 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.228720 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.228678 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bee772d4-84d4-43b8-8be9-26885da39d30-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-857b57b456-ns76c\" (UID: \"bee772d4-84d4-43b8-8be9-26885da39d30\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" Jul 15 16:59:37.228805 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.228785 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-ca\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.229017 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.228989 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/06f2e5cb-6674-46b8-919f-1b9b9b578683-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-76t88\" (UID: \"06f2e5cb-6674-46b8-919f-1b9b9b578683\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" Jul 15 16:59:37.229135 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.229092 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.229299 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.229282 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c7780bdf-d41c-4331-a92c-eb22aafb0e0c-klusterlet-config\") pod \"klusterlet-addon-workmgr-d66fd7f67-9jldr\" (UID: \"c7780bdf-d41c-4331-a92c-eb22aafb0e0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.239107 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.239085 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" Jul 15 16:59:37.242289 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.242262 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtsx9\" (UniqueName: \"kubernetes.io/projected/a9238314-bcd1-4ed3-b33c-3d0a11f8269b-kube-api-access-qtsx9\") pod \"cluster-proxy-proxy-agent-6c6f79f48d-6z76g\" (UID: \"a9238314-bcd1-4ed3-b33c-3d0a11f8269b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.243734 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.243713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhtht\" (UniqueName: \"kubernetes.io/projected/bee772d4-84d4-43b8-8be9-26885da39d30-kube-api-access-qhtht\") pod \"managed-serviceaccount-addon-agent-857b57b456-ns76c\" (UID: \"bee772d4-84d4-43b8-8be9-26885da39d30\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" Jul 15 16:59:37.244618 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.244581 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtx8w\" (UniqueName: \"kubernetes.io/projected/445343c7-3076-481a-8cf3-b83e5f6d66e8-kube-api-access-xtx8w\") pod \"network-check-source-869dcfcdcd-px4mw\" (UID: \"445343c7-3076-481a-8cf3-b83e5f6d66e8\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw" Jul 15 16:59:37.250889 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.250858 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7qx7\" (UniqueName: \"kubernetes.io/projected/c7780bdf-d41c-4331-a92c-eb22aafb0e0c-kube-api-access-m7qx7\") pod \"klusterlet-addon-workmgr-d66fd7f67-9jldr\" (UID: \"c7780bdf-d41c-4331-a92c-eb22aafb0e0c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.251485 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.251460 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnknl\" (UniqueName: \"kubernetes.io/projected/cdeee4e8-f5bd-466e-94da-e7934bab2620-kube-api-access-bnknl\") pod \"downloads-68c9cbb497-xd962\" (UID: \"cdeee4e8-f5bd-466e-94da-e7934bab2620\") " pod="openshift-console/downloads-68c9cbb497-xd962" Jul 15 16:59:37.274947 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.274922 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" Jul 15 16:59:37.290093 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.290059 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" Jul 15 16:59:37.314693 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.314653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-xd962" Jul 15 16:59:37.324545 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.324506 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw" Jul 15 16:59:37.324826 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.324806 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mlh5\" (UniqueName: \"kubernetes.io/projected/055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe-kube-api-access-2mlh5\") pod \"migrator-7f59b54f6c-l9pf5\" (UID: \"055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" Jul 15 16:59:37.334194 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:37.334169 2566 projected.go:291] Couldn't get configMap openshift-kube-storage-version-migrator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Jul 15 16:59:37.334306 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:37.334198 2566 projected.go:196] Error preparing data for projected volume kube-api-access-2mlh5 for pod openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5: configmap "openshift-service-ca.crt" not found Jul 15 16:59:37.334306 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:37.334270 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe-kube-api-access-2mlh5 podName:055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe nodeName:}" failed. No retries permitted until 2026-07-15 16:59:37.834250142 +0000 UTC m=+72.147955019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2mlh5" (UniqueName: "kubernetes.io/projected/055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe-kube-api-access-2mlh5") pod "migrator-7f59b54f6c-l9pf5" (UID: "055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe") : configmap "openshift-service-ca.crt" not found Jul 15 16:59:37.485224 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.485139 2566 patch_prober.go:28] interesting pod/image-registry-756d85f79-ct589 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 16:59:37.485356 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.485212 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-756d85f79-ct589" podUID="83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 16:59:37.550276 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.550234 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:37.930388 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.930350 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mlh5\" (UniqueName: \"kubernetes.io/projected/055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe-kube-api-access-2mlh5\") pod \"migrator-7f59b54f6c-l9pf5\" (UID: \"055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" Jul 15 16:59:37.933102 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.933074 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mlh5\" (UniqueName: \"kubernetes.io/projected/055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe-kube-api-access-2mlh5\") pod \"migrator-7f59b54f6c-l9pf5\" (UID: \"055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" Jul 15 16:59:37.942861 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:37.942834 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" Jul 15 16:59:39.597712 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:39.597681 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 16:59:40.240388 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.240358 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68f88567cf-6jgpt"] Jul 15 16:59:40.257407 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.257377 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.262523 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.262496 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 16:59:40.263779 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.263581 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 16:59:40.263779 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.263638 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 16:59:40.263779 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.263648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 16:59:40.264860 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.264823 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 16:59:40.265669 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.265646 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f88567cf-6jgpt"] Jul 15 16:59:40.269896 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.269876 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-g6j9q\"" Jul 15 16:59:40.350324 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.350289 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-oauth-serving-cert\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.350551 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.350328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-config\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.350551 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.350350 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-oauth-config\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.350551 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.350370 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-serving-cert\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.350551 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.350426 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-service-ca\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.350551 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.350487 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltsh9\" (UniqueName: \"kubernetes.io/projected/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-kube-api-access-ltsh9\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.451681 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.451640 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-oauth-serving-cert\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.451883 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.451696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-config\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.451883 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.451723 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-oauth-config\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.451883 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.451749 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-serving-cert\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.451883 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.451770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-service-ca\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.451883 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.451812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltsh9\" (UniqueName: \"kubernetes.io/projected/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-kube-api-access-ltsh9\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.452456 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.452431 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-oauth-serving-cert\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.452540 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.452515 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-config\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.453474 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.453447 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-service-ca\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.455372 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.455353 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-serving-cert\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.455459 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.455375 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-oauth-config\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.465389 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.465363 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltsh9\" (UniqueName: \"kubernetes.io/projected/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-kube-api-access-ltsh9\") pod \"console-68f88567cf-6jgpt\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:40.567457 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:40.567369 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 16:59:42.056263 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.056185 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c"] Jul 15 16:59:42.063280 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:42.063253 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbee772d4_84d4_43b8_8be9_26885da39d30.slice/crio-f229c212f90269050223614d95062efceebee432fe7230ae1c47df618791000b WatchSource:0}: Error finding container f229c212f90269050223614d95062efceebee432fe7230ae1c47df618791000b: Status 404 returned error can't find the container with id f229c212f90269050223614d95062efceebee432fe7230ae1c47df618791000b Jul 15 16:59:42.175335 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.175308 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f88567cf-6jgpt"] Jul 15 16:59:42.178224 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:42.178193 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16fc6d3_9808_4ae6_b601_2260e2bea6a6.slice/crio-dd6097c316e44eef52fff92e27447bdf59c2ed2d2dd19b3e03badc0970a1e851 WatchSource:0}: Error finding container dd6097c316e44eef52fff92e27447bdf59c2ed2d2dd19b3e03badc0970a1e851: Status 404 returned error can't find the container with id dd6097c316e44eef52fff92e27447bdf59c2ed2d2dd19b3e03badc0970a1e851 Jul 15 16:59:42.295110 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.295030 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77f756c57b-rjwfx"] Jul 15 16:59:42.298212 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.298190 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.302769 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:42.302735 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"console-77f756c57b-rjwfx\" is forbidden: User \"system:node:ip-10-0-143-124.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-143-124.ec2.internal' and this object" podUID="3f5b3213-8602-4f65-ba96-ebead7b1a585" pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.302874 ip-10-0-143-124 kubenswrapper[2566]: E0715 16:59:42.302760 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca-bundle\" is forbidden: User \"system:node:ip-10-0-143-124.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-console\": no relationship found between node 'ip-10-0-143-124.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" type="*v1.ConfigMap" Jul 15 16:59:42.321367 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:42.321327 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod445343c7_3076_481a_8cf3_b83e5f6d66e8.slice/crio-c5f65d5a61f59c17ee63617260902b7baefbe1afed6c062c24f263b6e35603d6 WatchSource:0}: Error finding container c5f65d5a61f59c17ee63617260902b7baefbe1afed6c062c24f263b6e35603d6: Status 404 returned error can't find the container with id c5f65d5a61f59c17ee63617260902b7baefbe1afed6c062c24f263b6e35603d6 Jul 15 16:59:42.325095 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:42.322309 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9238314_bcd1_4ed3_b33c_3d0a11f8269b.slice/crio-09f91514f13c0b5dc4e839e74df33881401a76d9f35451f2a3868fd425b864a7 WatchSource:0}: Error finding container 09f91514f13c0b5dc4e839e74df33881401a76d9f35451f2a3868fd425b864a7: Status 404 returned error can't find the container with id 09f91514f13c0b5dc4e839e74df33881401a76d9f35451f2a3868fd425b864a7 Jul 15 16:59:42.325095 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.323073 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g"] Jul 15 16:59:42.325095 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.324410 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw"] Jul 15 16:59:42.326015 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.325997 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77f756c57b-rjwfx"] Jul 15 16:59:42.347257 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.347232 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-xd962"] Jul 15 16:59:42.348569 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.348549 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-76t88"] Jul 15 16:59:42.349522 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.349505 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5"] Jul 15 16:59:42.369104 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.369063 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr"] Jul 15 16:59:42.372565 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:42.371435 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod055b8ea2_ffc0_4a22_bee4_6ef4d9f9dbbe.slice/crio-f26bc8225768fdfdbaa95917231c4624b4a8321f077746f7715d80dc530ed25f WatchSource:0}: Error finding container f26bc8225768fdfdbaa95917231c4624b4a8321f077746f7715d80dc530ed25f: Status 404 returned error can't find the container with id f26bc8225768fdfdbaa95917231c4624b4a8321f077746f7715d80dc530ed25f Jul 15 16:59:42.372847 ip-10-0-143-124 kubenswrapper[2566]: W0715 16:59:42.372785 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdeee4e8_f5bd_466e_94da_e7934bab2620.slice/crio-b53e04d2c0e477db270e0e42db11c6d41d420ee33a408e0827b7ec074ed67d4c WatchSource:0}: Error finding container b53e04d2c0e477db270e0e42db11c6d41d420ee33a408e0827b7ec074ed67d4c: Status 404 returned error can't find the container with id b53e04d2c0e477db270e0e42db11c6d41d420ee33a408e0827b7ec074ed67d4c Jul 15 16:59:42.372908 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.372859 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-trusted-ca-bundle\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.372964 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.372904 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-serving-cert\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.372964 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.372936 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-oauth-config\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.373065 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.372963 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-service-ca\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.373065 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.373014 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-oauth-serving-cert\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.373065 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.373053 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-config\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.373209 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.373103 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbmkh\" (UniqueName: \"kubernetes.io/projected/3f5b3213-8602-4f65-ba96-ebead7b1a585-kube-api-access-bbmkh\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.473513 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.473482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-oauth-serving-cert\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.473753 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.473532 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-config\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.473753 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.473568 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbmkh\" (UniqueName: \"kubernetes.io/projected/3f5b3213-8602-4f65-ba96-ebead7b1a585-kube-api-access-bbmkh\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.473753 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.473638 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-trusted-ca-bundle\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.473753 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.473663 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-serving-cert\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.473753 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.473685 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-oauth-config\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.473753 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.473708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-service-ca\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.474445 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.474389 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-config\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.474575 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.474468 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-service-ca\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.474575 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.474399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-oauth-serving-cert\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.476578 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.476551 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-serving-cert\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.476692 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.476656 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-oauth-config\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.487357 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.487335 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbmkh\" (UniqueName: \"kubernetes.io/projected/3f5b3213-8602-4f65-ba96-ebead7b1a585-kube-api-access-bbmkh\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:42.731081 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.731040 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d48rf" event={"ID":"7c43931a-91df-49ea-bea3-10e907fe1b5c","Type":"ContainerStarted","Data":"6218d85d356810c1d59240f635d82d998b084bcffee85378ff4bd41716588737"} Jul 15 16:59:42.737680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.737642 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" event={"ID":"06f2e5cb-6674-46b8-919f-1b9b9b578683","Type":"ContainerStarted","Data":"16e7d03fce46685f4fc986480230baaea20c6250cdf2e57db546f0405d8a7811"} Jul 15 16:59:42.743559 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.742875 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4nrmg" event={"ID":"f96fcf3d-4d29-4a71-a191-cb0b609a1602","Type":"ContainerStarted","Data":"a4fbb675353b68eb5bdb227b383f7f0f33e2594c8726b86a1ee04db141ab6c62"} Jul 15 16:59:42.746816 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.746784 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" event={"ID":"055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe","Type":"ContainerStarted","Data":"f26bc8225768fdfdbaa95917231c4624b4a8321f077746f7715d80dc530ed25f"} Jul 15 16:59:42.751132 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.751101 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" event={"ID":"c7780bdf-d41c-4331-a92c-eb22aafb0e0c","Type":"ContainerStarted","Data":"1488ccdac5e0d474903805f4efbfec0eb27a8072f7a63ba68a602ea12a050e9a"} Jul 15 16:59:42.756296 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.756261 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m4wq5" event={"ID":"fc473a95-bf8d-46ac-b047-8be9d1c1efc3","Type":"ContainerStarted","Data":"0df42725d17ec736f3c5bf56e27e189e8c8585f02ddfb1d199b8e3ea680349cb"} Jul 15 16:59:42.756979 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.756875 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 16:59:42.759536 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.759509 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-xd962" event={"ID":"cdeee4e8-f5bd-466e-94da-e7934bab2620","Type":"ContainerStarted","Data":"b53e04d2c0e477db270e0e42db11c6d41d420ee33a408e0827b7ec074ed67d4c"} Jul 15 16:59:42.763424 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.763397 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw" event={"ID":"445343c7-3076-481a-8cf3-b83e5f6d66e8","Type":"ContainerStarted","Data":"c186a6af3312348ff39f353a3987e8bd72d5d9cb5ab3bc805afca0bc9e7d5918"} Jul 15 16:59:42.763524 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.763447 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw" event={"ID":"445343c7-3076-481a-8cf3-b83e5f6d66e8","Type":"ContainerStarted","Data":"c5f65d5a61f59c17ee63617260902b7baefbe1afed6c062c24f263b6e35603d6"} Jul 15 16:59:42.769355 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.769306 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" event={"ID":"8c9266b8-1422-4e7a-9736-89316363fc85","Type":"ContainerStarted","Data":"b08f63fb828a5cc9dbd3a4c8a1b9c433bc5bc843dd88f14f584016c379882d5d"} Jul 15 16:59:42.771501 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.771450 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" event={"ID":"bee772d4-84d4-43b8-8be9-26885da39d30","Type":"ContainerStarted","Data":"f229c212f90269050223614d95062efceebee432fe7230ae1c47df618791000b"} Jul 15 16:59:42.773579 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.773523 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f88567cf-6jgpt" event={"ID":"f16fc6d3-9808-4ae6-b601-2260e2bea6a6","Type":"ContainerStarted","Data":"dd6097c316e44eef52fff92e27447bdf59c2ed2d2dd19b3e03badc0970a1e851"} Jul 15 16:59:42.776150 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.776121 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" event={"ID":"a9238314-bcd1-4ed3-b33c-3d0a11f8269b","Type":"ContainerStarted","Data":"09f91514f13c0b5dc4e839e74df33881401a76d9f35451f2a3868fd425b864a7"} Jul 15 16:59:42.792326 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.791535 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d48rf" podStartSLOduration=68.859365267 podStartE2EDuration="1m16.791512867s" totalImagesPullingTime="7.9321476s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:42.791327192 +0000 UTC m=+77.105032084" watchObservedRunningTime="2026-07-15 16:59:42.791512867 +0000 UTC m=+77.105217757" Jul 15 16:59:42.882300 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.881971 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-wd9bp" podStartSLOduration=41.140546781 podStartE2EDuration="48.881948042s" totalImagesPullingTime="7.741401261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:42.879973595 +0000 UTC m=+77.193678489" watchObservedRunningTime="2026-07-15 16:59:42.881948042 +0000 UTC m=+77.195652936" Jul 15 16:59:42.929402 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.928309 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-px4mw" podStartSLOduration=6.928287707 podStartE2EDuration="6.928287707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:42.925188351 +0000 UTC m=+77.238893245" watchObservedRunningTime="2026-07-15 16:59:42.928287707 +0000 UTC m=+77.241992601" Jul 15 16:59:42.967597 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:42.967534 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m4wq5" podStartSLOduration=69.100131855 podStartE2EDuration="1m16.967514157s" totalImagesPullingTime="7.867382302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:58:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:42.966053534 +0000 UTC m=+77.279758427" watchObservedRunningTime="2026-07-15 16:59:42.967514157 +0000 UTC m=+77.281219051" Jul 15 16:59:43.001972 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:43.001866 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4nrmg" podStartSLOduration=18.333388014 podStartE2EDuration="35.001842373s" totalImagesPullingTime="16.668454359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:59:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:42.995905815 +0000 UTC m=+77.309610705" watchObservedRunningTime="2026-07-15 16:59:43.001842373 +0000 UTC m=+77.315547269" Jul 15 16:59:43.261175 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:43.260890 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 16:59:43.264860 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:43.264802 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-trusted-ca-bundle\") pod \"console-77f756c57b-rjwfx\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:43.507701 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:43.507367 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 16:59:43.833088 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:43.832934 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77f756c57b-rjwfx"] Jul 15 16:59:44.846228 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:44.831986 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77f756c57b-rjwfx" event={"ID":"3f5b3213-8602-4f65-ba96-ebead7b1a585","Type":"ContainerStarted","Data":"ad34f8020e2704abd27750b60b341b0b115e04de3656dd89027003eafc72ce5a"} Jul 15 16:59:52.892022 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:52.891979 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" event={"ID":"06f2e5cb-6674-46b8-919f-1b9b9b578683","Type":"ContainerStarted","Data":"9af9d0d537389c39eefcdf8a131de1390b2bf6bb16520a35ae843ac3b701c419"} Jul 15 16:59:52.911799 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:52.911300 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" event={"ID":"055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe","Type":"ContainerStarted","Data":"1f360412d0286f84a3bde25296c11a3cf18b8e18dfcf6ce4cbfe218c7b3ef7b2"} Jul 15 16:59:52.927349 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:52.927230 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-76t88" podStartSLOduration=6.725757085 podStartE2EDuration="16.927199411s" totalImagesPullingTime="10.201442326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:52.926509324 +0000 UTC m=+87.240214218" watchObservedRunningTime="2026-07-15 16:59:52.927199411 +0000 UTC m=+87.240904305" Jul 15 16:59:53.919365 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.919328 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" event={"ID":"055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe","Type":"ContainerStarted","Data":"3ff2f9d7ec1731281cc02dcfb3b8b8a096e24e79e8aaafe554af9311bd3c10fa"} Jul 15 16:59:53.921497 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.921465 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" event={"ID":"c7780bdf-d41c-4331-a92c-eb22aafb0e0c","Type":"ContainerStarted","Data":"ed8f74f51314b10b92cf439a12b04dec82c02311ce208673aa7e9039891a3246"} Jul 15 16:59:53.921680 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.921665 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:53.923208 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.923176 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77f756c57b-rjwfx" event={"ID":"3f5b3213-8602-4f65-ba96-ebead7b1a585","Type":"ContainerStarted","Data":"e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1"} Jul 15 16:59:53.923950 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.923904 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" Jul 15 16:59:53.925005 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.924977 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" event={"ID":"bee772d4-84d4-43b8-8be9-26885da39d30","Type":"ContainerStarted","Data":"192277b0766a3c6a720d241c8d62dcaf1a1a97f6781a595d65fdde14f5a204bd"} Jul 15 16:59:53.926320 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.926301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f88567cf-6jgpt" event={"ID":"f16fc6d3-9808-4ae6-b601-2260e2bea6a6","Type":"ContainerStarted","Data":"f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74"} Jul 15 16:59:53.928057 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.928038 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" event={"ID":"a9238314-bcd1-4ed3-b33c-3d0a11f8269b","Type":"ContainerStarted","Data":"b95989316b1bddb11947b4215ca6cb53a0f9aff91bc7f4f59d5d5ea80f0f131c"} Jul 15 16:59:53.928148 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.928065 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" event={"ID":"a9238314-bcd1-4ed3-b33c-3d0a11f8269b","Type":"ContainerStarted","Data":"2021cb5a25d5bb5a08dc27996976ca8a0a5ca3dce9882110308fd53d5221fb15"} Jul 15 16:59:53.928148 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.928077 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" event={"ID":"a9238314-bcd1-4ed3-b33c-3d0a11f8269b","Type":"ContainerStarted","Data":"4ceed7d0499c56398b870dacb08da19ce34cdc909a09592d711b1e20309c6e2f"} Jul 15 16:59:53.939933 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.939889 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-l9pf5" podStartSLOduration=7.697616304 podStartE2EDuration="17.9398775s" totalImagesPullingTime="10.242261196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:53.938796658 +0000 UTC m=+88.252501550" watchObservedRunningTime="2026-07-15 16:59:53.9398775 +0000 UTC m=+88.253582370" Jul 15 16:59:53.964042 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.964001 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f79f48d-6z76g" podStartSLOduration=7.549973036 podStartE2EDuration="17.963987387s" totalImagesPullingTime="10.414014351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:53.962179027 +0000 UTC m=+88.275883918" watchObservedRunningTime="2026-07-15 16:59:53.963987387 +0000 UTC m=+88.277692257" Jul 15 16:59:53.985541 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:53.985482 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68f88567cf-6jgpt" podStartSLOduration=3.430138742 podStartE2EDuration="13.985467326s" totalImagesPullingTime="10.555328584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:53.984280361 +0000 UTC m=+88.297985255" watchObservedRunningTime="2026-07-15 16:59:53.985467326 +0000 UTC m=+88.299172218" Jul 15 16:59:54.005451 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:54.005406 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-d66fd7f67-9jldr" podStartSLOduration=7.645036028 podStartE2EDuration="18.005390775s" totalImagesPullingTime="10.360354747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:54.004415886 +0000 UTC m=+88.318120778" watchObservedRunningTime="2026-07-15 16:59:54.005390775 +0000 UTC m=+88.319095666" Jul 15 16:59:54.025098 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:54.025051 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-857b57b456-ns76c" podStartSLOduration=7.354489114 podStartE2EDuration="18.025035859s" totalImagesPullingTime="10.670546745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:54.023659992 +0000 UTC m=+88.337364886" watchObservedRunningTime="2026-07-15 16:59:54.025035859 +0000 UTC m=+88.338740751" Jul 15 16:59:54.050675 ip-10-0-143-124 kubenswrapper[2566]: I0715 16:59:54.050594 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77f756c57b-rjwfx" podStartSLOduration=3.141886963 podStartE2EDuration="12.050571028s" totalImagesPullingTime="8.908684065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:59:54.049371411 +0000 UTC m=+88.363076302" watchObservedRunningTime="2026-07-15 16:59:54.050571028 +0000 UTC m=+88.364275900" Jul 15 17:00:00.567651 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:00.567597 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 17:00:00.568081 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:00.567704 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 17:00:00.574284 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:00.574240 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 17:00:00.861642 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:00.860229 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f9jw8"] Jul 15 17:00:01.003691 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.003651 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 17:00:01.003877 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.003799 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.009455 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.009252 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 17:00:01.011250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.010951 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rnszz\"" Jul 15 17:00:01.011250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.011002 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 17:00:01.011250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.011047 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 17:00:01.011250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.010955 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 17:00:01.102286 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.102231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a16dca-6c39-4e50-9313-cda811d5395e-sys\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.103096 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.102527 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.103096 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.102571 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11a16dca-6c39-4e50-9313-cda811d5395e-metrics-client-ca\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.103096 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.102634 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-textfile\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.103096 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.102852 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11a16dca-6c39-4e50-9313-cda811d5395e-root\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.103096 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.102887 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-accelerators-collector-config\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.103096 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.102949 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-wtmp\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.103096 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.102995 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-tls\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.103096 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.103024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmjk7\" (UniqueName: \"kubernetes.io/projected/11a16dca-6c39-4e50-9313-cda811d5395e-kube-api-access-zmjk7\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.204075 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.203988 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11a16dca-6c39-4e50-9313-cda811d5395e-root\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.204075 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204029 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-accelerators-collector-config\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.204075 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-wtmp\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.204342 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204093 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-tls\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.204342 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmjk7\" (UniqueName: \"kubernetes.io/projected/11a16dca-6c39-4e50-9313-cda811d5395e-kube-api-access-zmjk7\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.204342 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204177 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a16dca-6c39-4e50-9313-cda811d5395e-sys\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.204342 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.204342 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204234 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11a16dca-6c39-4e50-9313-cda811d5395e-metrics-client-ca\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.204342 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-textfile\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.205004 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204589 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-textfile\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.205004 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204696 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/11a16dca-6c39-4e50-9313-cda811d5395e-root\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.205004 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204712 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11a16dca-6c39-4e50-9313-cda811d5395e-sys\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.205004 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.204833 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-wtmp\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.205263 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.205242 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-accelerators-collector-config\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.205630 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.205582 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11a16dca-6c39-4e50-9313-cda811d5395e-metrics-client-ca\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.207444 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.207421 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.207580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.207554 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/11a16dca-6c39-4e50-9313-cda811d5395e-node-exporter-tls\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.217729 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.217434 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmjk7\" (UniqueName: \"kubernetes.io/projected/11a16dca-6c39-4e50-9313-cda811d5395e-kube-api-access-zmjk7\") pod \"node-exporter-f9jw8\" (UID: \"11a16dca-6c39-4e50-9313-cda811d5395e\") " pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:01.317695 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:01.317660 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f9jw8" Jul 15 17:00:03.023418 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:00:03.023379 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a16dca_6c39_4e50_9313_cda811d5395e.slice/crio-40ba48f81ddddd30b745f6af8406f4011ac8697a6380df30092d9ead8331ab4d WatchSource:0}: Error finding container 40ba48f81ddddd30b745f6af8406f4011ac8697a6380df30092d9ead8331ab4d: Status 404 returned error can't find the container with id 40ba48f81ddddd30b745f6af8406f4011ac8697a6380df30092d9ead8331ab4d Jul 15 17:00:03.508655 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:03.508598 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 17:00:03.509149 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:03.508715 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 17:00:03.513785 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:03.513760 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 17:00:03.979932 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:03.979897 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f9jw8" event={"ID":"11a16dca-6c39-4e50-9313-cda811d5395e","Type":"ContainerStarted","Data":"40ba48f81ddddd30b745f6af8406f4011ac8697a6380df30092d9ead8331ab4d"} Jul 15 17:00:03.983321 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:03.983290 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-xd962" event={"ID":"cdeee4e8-f5bd-466e-94da-e7934bab2620","Type":"ContainerStarted","Data":"a82019615452a20541ba48072b59e58c65699dd8b92d13f372ac0f271c53ca2d"} Jul 15 17:00:03.985507 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:03.985459 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-xd962" Jul 15 17:00:03.988021 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:03.987996 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 17:00:03.994130 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:03.994107 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-xd962" Jul 15 17:00:04.015651 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:04.014817 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-xd962" podStartSLOduration=7.052181001 podStartE2EDuration="28.014797804s" totalImagesPullingTime="20.962616803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:04.013189832 +0000 UTC m=+98.326894738" watchObservedRunningTime="2026-07-15 17:00:04.014797804 +0000 UTC m=+98.328502697" Jul 15 17:00:04.123707 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:04.122882 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f88567cf-6jgpt"] Jul 15 17:00:04.704953 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:04.704920 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-756d85f79-ct589"] Jul 15 17:00:05.999626 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:05.997872 2566 generic.go:356] "Generic (PLEG): container finished" podID="11a16dca-6c39-4e50-9313-cda811d5395e" containerID="e92df26652662dbdbcc575747692bb962961616d1fcba2982ae357b741b46824" exitCode=0 Jul 15 17:00:05.999626 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:05.998718 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f9jw8" event={"ID":"11a16dca-6c39-4e50-9313-cda811d5395e","Type":"ContainerDied","Data":"e92df26652662dbdbcc575747692bb962961616d1fcba2982ae357b741b46824"} Jul 15 17:00:07.006049 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:07.006010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f9jw8" event={"ID":"11a16dca-6c39-4e50-9313-cda811d5395e","Type":"ContainerStarted","Data":"4d35a8112405944a27a2354e8ec4bc80dad876f8c17a6eb80acd04ad8bd272eb"} Jul 15 17:00:07.006484 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:07.006073 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f9jw8" event={"ID":"11a16dca-6c39-4e50-9313-cda811d5395e","Type":"ContainerStarted","Data":"41b7f11d0a969f386e4d51801106e8d5011f326e856ac5b5632f9bf75770bc8b"} Jul 15 17:00:07.032344 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:07.032258 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f9jw8" podStartSLOduration=4.958423106 podStartE2EDuration="7.032235883s" totalImagesPullingTime="1.938271601s" totalInitContainerRuntime="135.541176ms" isStatefulPod=true podCreationTimestamp="2026-07-15 17:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:07.029749236 +0000 UTC m=+101.343454128" watchObservedRunningTime="2026-07-15 17:00:07.032235883 +0000 UTC m=+101.345940776" Jul 15 17:00:13.036261 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:13.036217 2566 generic.go:356] "Generic (PLEG): container finished" podID="1c975720-099d-446b-bcf7-cb34eb853f8a" containerID="39026a8175e4f8763475f874b5c5249e58abf025ed05b07df215b64dc2984528" exitCode=0 Jul 15 17:00:13.036261 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:13.036262 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" event={"ID":"1c975720-099d-446b-bcf7-cb34eb853f8a","Type":"ContainerDied","Data":"39026a8175e4f8763475f874b5c5249e58abf025ed05b07df215b64dc2984528"} Jul 15 17:00:13.036866 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:13.036561 2566 scope.go:117] "RemoveContainer" containerID="39026a8175e4f8763475f874b5c5249e58abf025ed05b07df215b64dc2984528" Jul 15 17:00:14.042350 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.042295 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-8z7k5" event={"ID":"1c975720-099d-446b-bcf7-cb34eb853f8a","Type":"ContainerStarted","Data":"7cf10140fa04d9a404acc23cbe216f54ade2236b6796027f72f370f26e4fd3eb"} Jul 15 17:00:14.834793 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.834763 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m4wq5" Jul 15 17:00:14.837047 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.837019 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76f79b68d6-w4qpv"] Jul 15 17:00:14.903712 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.903677 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:14.930525 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.930492 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-serving-cert\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:14.930716 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.930645 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-oauth-serving-cert\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:14.930716 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.930677 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-service-ca\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:14.930814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.930720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-trusted-ca-bundle\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:14.930814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.930744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-config\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:14.930814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.930790 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x257h\" (UniqueName: \"kubernetes.io/projected/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-kube-api-access-x257h\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:14.930953 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.930908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-oauth-config\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:14.936761 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:14.936734 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76f79b68d6-w4qpv"] Jul 15 17:00:15.031717 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.031677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-serving-cert\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.031903 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.031805 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-oauth-serving-cert\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.031903 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.031837 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-service-ca\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.031903 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.031873 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-trusted-ca-bundle\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.031903 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.031896 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-config\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.032119 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.031921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x257h\" (UniqueName: \"kubernetes.io/projected/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-kube-api-access-x257h\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.032119 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.031984 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-oauth-config\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.032953 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.032924 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-trusted-ca-bundle\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.033124 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.033011 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-config\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.033254 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.033218 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-oauth-serving-cert\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.033481 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.033458 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-service-ca\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.035180 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.035147 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-serving-cert\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.035553 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.035531 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-oauth-config\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.062240 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.062210 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x257h\" (UniqueName: \"kubernetes.io/projected/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-kube-api-access-x257h\") pod \"console-76f79b68d6-w4qpv\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.212938 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.212836 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:15.381869 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:15.381789 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76f79b68d6-w4qpv"] Jul 15 17:00:15.384462 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:00:15.384434 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a83cbc4_29ec_4254_90d4_4cfb8a55ac9b.slice/crio-14d7759ffbb23e977ce007172a74de628860d1f3535d5f3ed5ca2d33890a1e92 WatchSource:0}: Error finding container 14d7759ffbb23e977ce007172a74de628860d1f3535d5f3ed5ca2d33890a1e92: Status 404 returned error can't find the container with id 14d7759ffbb23e977ce007172a74de628860d1f3535d5f3ed5ca2d33890a1e92 Jul 15 17:00:16.060447 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:16.060409 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f79b68d6-w4qpv" event={"ID":"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b","Type":"ContainerStarted","Data":"34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8"} Jul 15 17:00:16.060653 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:16.060459 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f79b68d6-w4qpv" event={"ID":"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b","Type":"ContainerStarted","Data":"14d7759ffbb23e977ce007172a74de628860d1f3535d5f3ed5ca2d33890a1e92"} Jul 15 17:00:18.070639 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:18.070532 2566 generic.go:356] "Generic (PLEG): container finished" podID="63bf6178-a18e-4b9c-81bb-98a008e87fc8" containerID="62beab362460c058e54969e85c776ef7d2999a4641b4925fd87b0d44f35a4a4e" exitCode=0 Jul 15 17:00:18.070639 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:18.070596 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" event={"ID":"63bf6178-a18e-4b9c-81bb-98a008e87fc8","Type":"ContainerDied","Data":"62beab362460c058e54969e85c776ef7d2999a4641b4925fd87b0d44f35a4a4e"} Jul 15 17:00:18.071182 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:18.071031 2566 scope.go:117] "RemoveContainer" containerID="62beab362460c058e54969e85c776ef7d2999a4641b4925fd87b0d44f35a4a4e" Jul 15 17:00:18.103990 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:18.103948 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76f79b68d6-w4qpv" podStartSLOduration=4.103932525 podStartE2EDuration="4.103932525s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:00:16.090028631 +0000 UTC m=+110.403733522" watchObservedRunningTime="2026-07-15 17:00:18.103932525 +0000 UTC m=+112.417637419" Jul 15 17:00:19.076835 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:19.076798 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-nzcjg" event={"ID":"63bf6178-a18e-4b9c-81bb-98a008e87fc8","Type":"ContainerStarted","Data":"636954a3c6a0238965e2336860cf4ba81284502339b72492f508c0a9e87e53e7"} Jul 15 17:00:21.087165 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:21.087133 2566 generic.go:356] "Generic (PLEG): container finished" podID="8a3ef658-21e3-409d-973a-ef9afbbb3b46" containerID="4472ce4ac4b593f1f42b9e08996b8cf544aa827597b7e4a9c264a2bdc4e40f18" exitCode=0 Jul 15 17:00:21.087532 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:21.087209 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" event={"ID":"8a3ef658-21e3-409d-973a-ef9afbbb3b46","Type":"ContainerDied","Data":"4472ce4ac4b593f1f42b9e08996b8cf544aa827597b7e4a9c264a2bdc4e40f18"} Jul 15 17:00:21.087593 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:21.087579 2566 scope.go:117] "RemoveContainer" containerID="4472ce4ac4b593f1f42b9e08996b8cf544aa827597b7e4a9c264a2bdc4e40f18" Jul 15 17:00:22.093687 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:22.093655 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-qq9xb" event={"ID":"8a3ef658-21e3-409d-973a-ef9afbbb3b46","Type":"ContainerStarted","Data":"879700203c4a6165332771edc5719fc152303f98dae9d1920ddb6ee24b74a542"} Jul 15 17:00:25.213008 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:25.212968 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:25.213874 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:25.213068 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:25.218186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:25.218157 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:26.115213 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:26.115187 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:00:26.170890 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:26.170858 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77f756c57b-rjwfx"] Jul 15 17:00:29.156656 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.156590 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68f88567cf-6jgpt" podUID="f16fc6d3-9808-4ae6-b601-2260e2bea6a6" containerName="console" containerID="cri-o://f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74" gracePeriod=15 Jul 15 17:00:29.465672 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.465646 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 17:00:29.541848 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.541816 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-config\") pod \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " Jul 15 17:00:29.542018 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.541870 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-oauth-serving-cert\") pod \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " Jul 15 17:00:29.542018 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.541966 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-service-ca\") pod \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " Jul 15 17:00:29.542018 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.541991 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltsh9\" (UniqueName: \"kubernetes.io/projected/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-kube-api-access-ltsh9\") pod \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " Jul 15 17:00:29.542018 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.542008 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-oauth-config\") pod \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " Jul 15 17:00:29.542216 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.542036 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-serving-cert\") pod \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\" (UID: \"f16fc6d3-9808-4ae6-b601-2260e2bea6a6\") " Jul 15 17:00:29.542275 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.542242 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-config" (OuterVolumeSpecName: "console-config") pod "f16fc6d3-9808-4ae6-b601-2260e2bea6a6" (UID: "f16fc6d3-9808-4ae6-b601-2260e2bea6a6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:00:29.542331 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.542301 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f16fc6d3-9808-4ae6-b601-2260e2bea6a6" (UID: "f16fc6d3-9808-4ae6-b601-2260e2bea6a6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:00:29.542385 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.542353 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-service-ca" (OuterVolumeSpecName: "service-ca") pod "f16fc6d3-9808-4ae6-b601-2260e2bea6a6" (UID: "f16fc6d3-9808-4ae6-b601-2260e2bea6a6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:00:29.544679 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.544642 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f16fc6d3-9808-4ae6-b601-2260e2bea6a6" (UID: "f16fc6d3-9808-4ae6-b601-2260e2bea6a6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:00:29.544679 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.544654 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-kube-api-access-ltsh9" (OuterVolumeSpecName: "kube-api-access-ltsh9") pod "f16fc6d3-9808-4ae6-b601-2260e2bea6a6" (UID: "f16fc6d3-9808-4ae6-b601-2260e2bea6a6"). InnerVolumeSpecName "kube-api-access-ltsh9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:00:29.544877 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.544698 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f16fc6d3-9808-4ae6-b601-2260e2bea6a6" (UID: "f16fc6d3-9808-4ae6-b601-2260e2bea6a6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:00:29.642873 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.642842 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-config\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:29.642873 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.642871 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-oauth-serving-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:29.642873 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.642881 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-service-ca\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:29.643120 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.642889 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltsh9\" (UniqueName: \"kubernetes.io/projected/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-kube-api-access-ltsh9\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:29.643120 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.642900 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-oauth-config\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:29.643120 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.642908 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f16fc6d3-9808-4ae6-b601-2260e2bea6a6-console-serving-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:29.733087 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:29.732997 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-756d85f79-ct589" podUID="83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" containerName="registry" containerID="cri-o://b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6" gracePeriod=30 Jul 15 17:00:30.021964 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.021928 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 17:00:30.045717 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.045685 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-image-registry-private-configuration\") pod \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " Jul 15 17:00:30.045901 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.045728 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzlhh\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-kube-api-access-kzlhh\") pod \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " Jul 15 17:00:30.045901 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.045769 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-installation-pull-secrets\") pod \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " Jul 15 17:00:30.045901 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.045813 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") pod \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " Jul 15 17:00:30.045901 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.045867 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-ca-trust-extracted\") pod \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " Jul 15 17:00:30.045901 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.045894 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-certificates\") pod \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " Jul 15 17:00:30.046142 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.045950 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-trusted-ca\") pod \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " Jul 15 17:00:30.046142 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.046024 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-bound-sa-token\") pod \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\" (UID: \"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2\") " Jul 15 17:00:30.046394 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.046363 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:00:30.046703 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.046672 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:00:30.052503 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.052456 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-kube-api-access-kzlhh" (OuterVolumeSpecName: "kube-api-access-kzlhh") pod "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2"). InnerVolumeSpecName "kube-api-access-kzlhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:00:30.052503 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.052476 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:00:30.052724 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.052520 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:00:30.052783 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.052731 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:00:30.054076 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.054045 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:00:30.056777 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.056748 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" (UID: "83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:00:30.130105 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.130071 2566 generic.go:356] "Generic (PLEG): container finished" podID="f16fc6d3-9808-4ae6-b601-2260e2bea6a6" containerID="f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74" exitCode=0 Jul 15 17:00:30.130290 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.130134 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f88567cf-6jgpt" Jul 15 17:00:30.130290 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.130148 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f88567cf-6jgpt" event={"ID":"f16fc6d3-9808-4ae6-b601-2260e2bea6a6","Type":"ContainerDied","Data":"f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74"} Jul 15 17:00:30.130290 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.130200 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f88567cf-6jgpt" event={"ID":"f16fc6d3-9808-4ae6-b601-2260e2bea6a6","Type":"ContainerDied","Data":"dd6097c316e44eef52fff92e27447bdf59c2ed2d2dd19b3e03badc0970a1e851"} Jul 15 17:00:30.130290 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.130222 2566 scope.go:117] "RemoveContainer" containerID="f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74" Jul 15 17:00:30.131658 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.131632 2566 generic.go:356] "Generic (PLEG): container finished" podID="83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" containerID="b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6" exitCode=0 Jul 15 17:00:30.131776 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.131679 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-756d85f79-ct589" event={"ID":"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2","Type":"ContainerDied","Data":"b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6"} Jul 15 17:00:30.131776 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.131715 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-756d85f79-ct589" event={"ID":"83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2","Type":"ContainerDied","Data":"69ce7778f5546ff9153da063de62560275ba44049fa67ea83ad6aa4fa7cbe930"} Jul 15 17:00:30.131776 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.131738 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-756d85f79-ct589" Jul 15 17:00:30.147321 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.147279 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-image-registry-private-configuration\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:30.147321 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.147313 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzlhh\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-kube-api-access-kzlhh\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:30.147511 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.147329 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-installation-pull-secrets\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:30.147511 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.147342 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-tls\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:30.147511 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.147357 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-ca-trust-extracted\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:30.147511 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.147369 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-registry-certificates\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:30.147511 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.147382 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-trusted-ca\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:30.147511 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.147394 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2-bound-sa-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:30.149981 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.149906 2566 scope.go:117] "RemoveContainer" containerID="f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74" Jul 15 17:00:30.150501 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:00:30.150385 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74\": container with ID starting with f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74 not found: ID does not exist" containerID="f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74" Jul 15 17:00:30.150501 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.150430 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74"} err="failed to get container status \"f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74\": rpc error: code = NotFound desc = could not find container \"f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74\": container with ID starting with f5161597481f4106e752174e44e0f94b338777b9ce4e692d42b66b9929ad0d74 not found: ID does not exist" Jul 15 17:00:30.150501 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.150476 2566 scope.go:117] "RemoveContainer" containerID="b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6" Jul 15 17:00:30.160205 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.159949 2566 scope.go:117] "RemoveContainer" containerID="b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6" Jul 15 17:00:30.160547 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:00:30.160251 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6\": container with ID starting with b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6 not found: ID does not exist" containerID="b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6" Jul 15 17:00:30.160547 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.160286 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6"} err="failed to get container status \"b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6\": rpc error: code = NotFound desc = could not find container \"b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6\": container with ID starting with b398611e760764d313ef80e36000bcc8dba28f4ec5caf3c9f71bfd262866fab6 not found: ID does not exist" Jul 15 17:00:30.161182 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.161160 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-756d85f79-ct589"] Jul 15 17:00:30.169641 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.169591 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-756d85f79-ct589"] Jul 15 17:00:30.182213 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.182185 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f88567cf-6jgpt"] Jul 15 17:00:30.189153 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.189121 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" path="/var/lib/kubelet/pods/83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2/volumes" Jul 15 17:00:30.189423 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:30.189409 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68f88567cf-6jgpt"] Jul 15 17:00:32.187653 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:32.187591 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16fc6d3-9808-4ae6-b601-2260e2bea6a6" path="/var/lib/kubelet/pods/f16fc6d3-9808-4ae6-b601-2260e2bea6a6/volumes" Jul 15 17:00:51.197452 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.197407 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77f756c57b-rjwfx" podUID="3f5b3213-8602-4f65-ba96-ebead7b1a585" containerName="console" containerID="cri-o://e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1" gracePeriod=15 Jul 15 17:00:51.501461 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.501424 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 17:00:51.628237 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628202 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-oauth-config\") pod \"3f5b3213-8602-4f65-ba96-ebead7b1a585\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " Jul 15 17:00:51.628237 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628247 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-serving-cert\") pod \"3f5b3213-8602-4f65-ba96-ebead7b1a585\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " Jul 15 17:00:51.628486 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628297 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-oauth-serving-cert\") pod \"3f5b3213-8602-4f65-ba96-ebead7b1a585\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " Jul 15 17:00:51.628486 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628315 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-service-ca\") pod \"3f5b3213-8602-4f65-ba96-ebead7b1a585\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " Jul 15 17:00:51.628486 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628335 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-config\") pod \"3f5b3213-8602-4f65-ba96-ebead7b1a585\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " Jul 15 17:00:51.628486 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628459 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-trusted-ca-bundle\") pod \"3f5b3213-8602-4f65-ba96-ebead7b1a585\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " Jul 15 17:00:51.628707 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628516 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbmkh\" (UniqueName: \"kubernetes.io/projected/3f5b3213-8602-4f65-ba96-ebead7b1a585-kube-api-access-bbmkh\") pod \"3f5b3213-8602-4f65-ba96-ebead7b1a585\" (UID: \"3f5b3213-8602-4f65-ba96-ebead7b1a585\") " Jul 15 17:00:51.628814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628787 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3f5b3213-8602-4f65-ba96-ebead7b1a585" (UID: "3f5b3213-8602-4f65-ba96-ebead7b1a585"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:00:51.628869 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628794 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-service-ca" (OuterVolumeSpecName: "service-ca") pod "3f5b3213-8602-4f65-ba96-ebead7b1a585" (UID: "3f5b3213-8602-4f65-ba96-ebead7b1a585"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:00:51.628937 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.628881 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3f5b3213-8602-4f65-ba96-ebead7b1a585" (UID: "3f5b3213-8602-4f65-ba96-ebead7b1a585"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:00:51.629121 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.629086 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-config" (OuterVolumeSpecName: "console-config") pod "3f5b3213-8602-4f65-ba96-ebead7b1a585" (UID: "3f5b3213-8602-4f65-ba96-ebead7b1a585"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:00:51.630494 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.630465 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3f5b3213-8602-4f65-ba96-ebead7b1a585" (UID: "3f5b3213-8602-4f65-ba96-ebead7b1a585"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:00:51.630758 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.630739 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f5b3213-8602-4f65-ba96-ebead7b1a585-kube-api-access-bbmkh" (OuterVolumeSpecName: "kube-api-access-bbmkh") pod "3f5b3213-8602-4f65-ba96-ebead7b1a585" (UID: "3f5b3213-8602-4f65-ba96-ebead7b1a585"). InnerVolumeSpecName "kube-api-access-bbmkh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:00:51.630891 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.630874 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3f5b3213-8602-4f65-ba96-ebead7b1a585" (UID: "3f5b3213-8602-4f65-ba96-ebead7b1a585"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:00:51.729758 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.729676 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-trusted-ca-bundle\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:51.729758 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.729707 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbmkh\" (UniqueName: \"kubernetes.io/projected/3f5b3213-8602-4f65-ba96-ebead7b1a585-kube-api-access-bbmkh\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:51.729758 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.729717 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-oauth-config\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:51.729758 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.729725 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-serving-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:51.729758 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.729734 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-oauth-serving-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:51.729758 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.729742 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-service-ca\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:51.729758 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:51.729750 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f5b3213-8602-4f65-ba96-ebead7b1a585-console-config\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:00:52.230733 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:52.230699 2566 generic.go:356] "Generic (PLEG): container finished" podID="3f5b3213-8602-4f65-ba96-ebead7b1a585" containerID="e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1" exitCode=0 Jul 15 17:00:52.231193 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:52.230748 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77f756c57b-rjwfx" Jul 15 17:00:52.231193 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:52.230777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77f756c57b-rjwfx" event={"ID":"3f5b3213-8602-4f65-ba96-ebead7b1a585","Type":"ContainerDied","Data":"e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1"} Jul 15 17:00:52.231193 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:52.230823 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77f756c57b-rjwfx" event={"ID":"3f5b3213-8602-4f65-ba96-ebead7b1a585","Type":"ContainerDied","Data":"ad34f8020e2704abd27750b60b341b0b115e04de3656dd89027003eafc72ce5a"} Jul 15 17:00:52.231193 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:52.230845 2566 scope.go:117] "RemoveContainer" containerID="e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1" Jul 15 17:00:52.239510 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:52.239492 2566 scope.go:117] "RemoveContainer" containerID="e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1" Jul 15 17:00:52.239818 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:00:52.239793 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1\": container with ID starting with e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1 not found: ID does not exist" containerID="e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1" Jul 15 17:00:52.239896 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:52.239828 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1"} err="failed to get container status \"e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1\": rpc error: code = NotFound desc = could not find container \"e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1\": container with ID starting with e167aa2f9430464554b8979dee972253767eefe0fd54ac43a29dd7a41b3a27b1 not found: ID does not exist" Jul 15 17:00:52.250579 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:52.250552 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77f756c57b-rjwfx"] Jul 15 17:00:52.254038 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:52.254019 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77f756c57b-rjwfx"] Jul 15 17:00:54.187571 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:00:54.187532 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f5b3213-8602-4f65-ba96-ebead7b1a585" path="/var/lib/kubelet/pods/3f5b3213-8602-4f65-ba96-ebead7b1a585/volumes" Jul 15 17:01:42.086702 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:01:42.086631 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76f79b68d6-w4qpv"] Jul 15 17:02:07.108266 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.108194 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76f79b68d6-w4qpv" podUID="3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" containerName="console" containerID="cri-o://34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8" gracePeriod=15 Jul 15 17:02:07.359128 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.359066 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:02:07.434237 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434203 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-config\") pod \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " Jul 15 17:02:07.434422 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434259 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x257h\" (UniqueName: \"kubernetes.io/projected/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-kube-api-access-x257h\") pod \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " Jul 15 17:02:07.434422 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434283 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-serving-cert\") pod \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " Jul 15 17:02:07.434422 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434303 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-oauth-serving-cert\") pod \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " Jul 15 17:02:07.434422 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434324 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-oauth-config\") pod \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " Jul 15 17:02:07.434422 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434355 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-trusted-ca-bundle\") pod \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " Jul 15 17:02:07.434712 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434544 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-service-ca\") pod \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\" (UID: \"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b\") " Jul 15 17:02:07.434789 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434757 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-config" (OuterVolumeSpecName: "console-config") pod "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" (UID: "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:02:07.434888 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434860 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" (UID: "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:02:07.434888 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434879 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" (UID: "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:02:07.434995 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.434894 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-config\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:02:07.435083 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.435062 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-service-ca" (OuterVolumeSpecName: "service-ca") pod "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" (UID: "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:02:07.436809 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.436778 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" (UID: "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:02:07.436809 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.436788 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-kube-api-access-x257h" (OuterVolumeSpecName: "kube-api-access-x257h") pod "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" (UID: "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b"). InnerVolumeSpecName "kube-api-access-x257h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:02:07.436937 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.436793 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" (UID: "3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 17:02:07.535999 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.535972 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-oauth-config\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:02:07.535999 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.535994 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-trusted-ca-bundle\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:02:07.535999 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.536003 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-service-ca\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:02:07.536220 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.536012 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x257h\" (UniqueName: \"kubernetes.io/projected/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-kube-api-access-x257h\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:02:07.536220 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.536020 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-console-serving-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:02:07.536220 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.536028 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b-oauth-serving-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:02:07.537913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.537891 2566 generic.go:356] "Generic (PLEG): container finished" podID="3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" containerID="34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8" exitCode=0 Jul 15 17:02:07.538009 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.537924 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f79b68d6-w4qpv" event={"ID":"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b","Type":"ContainerDied","Data":"34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8"} Jul 15 17:02:07.538009 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.537946 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f79b68d6-w4qpv" Jul 15 17:02:07.538072 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.538008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f79b68d6-w4qpv" event={"ID":"3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b","Type":"ContainerDied","Data":"14d7759ffbb23e977ce007172a74de628860d1f3535d5f3ed5ca2d33890a1e92"} Jul 15 17:02:07.538072 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.538030 2566 scope.go:117] "RemoveContainer" containerID="34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8" Jul 15 17:02:07.546862 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.546841 2566 scope.go:117] "RemoveContainer" containerID="34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8" Jul 15 17:02:07.547136 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:02:07.547117 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8\": container with ID starting with 34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8 not found: ID does not exist" containerID="34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8" Jul 15 17:02:07.547184 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.547145 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8"} err="failed to get container status \"34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8\": rpc error: code = NotFound desc = could not find container \"34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8\": container with ID starting with 34a151475f3e554d357d3bc287c7dcea495f061409af46dfcfdebb34c5d807f8 not found: ID does not exist" Jul 15 17:02:07.559195 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.559173 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76f79b68d6-w4qpv"] Jul 15 17:02:07.563035 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:07.563013 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76f79b68d6-w4qpv"] Jul 15 17:02:08.186937 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:02:08.186903 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" path="/var/lib/kubelet/pods/3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b/volumes" Jul 15 17:03:26.085380 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:26.085351 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:03:26.091298 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:26.091263 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:03:26.092427 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:26.092403 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:03:26.096689 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:26.096670 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 17:03:26.098593 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:26.098574 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:03:44.999006 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.998974 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57"] Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999293 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f5b3213-8602-4f65-ba96-ebead7b1a585" containerName="console" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999304 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f5b3213-8602-4f65-ba96-ebead7b1a585" containerName="console" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999314 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" containerName="console" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999338 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" containerName="console" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999345 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" containerName="registry" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999350 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" containerName="registry" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999360 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16fc6d3-9808-4ae6-b601-2260e2bea6a6" containerName="console" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999366 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16fc6d3-9808-4ae6-b601-2260e2bea6a6" containerName="console" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999421 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a83cbc4-29ec-4254-90d4-4cfb8a55ac9b" containerName="console" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999429 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="83dd4c57-2b5b-4da1-8d69-a6a1e8652fb2" containerName="registry" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999434 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f5b3213-8602-4f65-ba96-ebead7b1a585" containerName="console" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:44.999442 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16fc6d3-9808-4ae6-b601-2260e2bea6a6" containerName="console" Jul 15 17:03:45.005251 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.003155 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.008188 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.008159 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-z4dlf\"" Jul 15 17:03:45.009137 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.009113 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 17:03:45.009247 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.009218 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 17:03:45.009297 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.009219 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 17:03:45.009428 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.009403 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 17:03:45.022496 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.022471 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57"] Jul 15 17:03:45.095668 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.095617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/29c867b6-c059-4d03-a672-5a5b3448f24a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.095668 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.095668 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/29c867b6-c059-4d03-a672-5a5b3448f24a-manifests\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.095910 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.095692 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/29c867b6-c059-4d03-a672-5a5b3448f24a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.095910 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.095715 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29c867b6-c059-4d03-a672-5a5b3448f24a-tmp\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.095910 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.095735 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p8ht\" (UniqueName: \"kubernetes.io/projected/29c867b6-c059-4d03-a672-5a5b3448f24a-kube-api-access-9p8ht\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.197179 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.197142 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/29c867b6-c059-4d03-a672-5a5b3448f24a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.197179 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.197178 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/29c867b6-c059-4d03-a672-5a5b3448f24a-manifests\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.197428 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.197202 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/29c867b6-c059-4d03-a672-5a5b3448f24a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.197428 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.197219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29c867b6-c059-4d03-a672-5a5b3448f24a-tmp\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.197428 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.197238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p8ht\" (UniqueName: \"kubernetes.io/projected/29c867b6-c059-4d03-a672-5a5b3448f24a-kube-api-access-9p8ht\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.197701 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.197663 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/29c867b6-c059-4d03-a672-5a5b3448f24a-manifests\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.199751 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.199719 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29c867b6-c059-4d03-a672-5a5b3448f24a-tmp\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.199922 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.199899 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/29c867b6-c059-4d03-a672-5a5b3448f24a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.199980 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.199916 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/29c867b6-c059-4d03-a672-5a5b3448f24a-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.205964 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.205943 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p8ht\" (UniqueName: \"kubernetes.io/projected/29c867b6-c059-4d03-a672-5a5b3448f24a-kube-api-access-9p8ht\") pod \"opendatahub-operator-controller-manager-7fdf764586-98m57\" (UID: \"29c867b6-c059-4d03-a672-5a5b3448f24a\") " pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.314650 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.314529 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:45.470207 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.470162 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57"] Jul 15 17:03:45.472586 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:03:45.472559 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c867b6_c059_4d03_a672_5a5b3448f24a.slice/crio-4990bd6611f98307bf1feb2bc87117fe900f3e036a34780250b8aad44bf3c512 WatchSource:0}: Error finding container 4990bd6611f98307bf1feb2bc87117fe900f3e036a34780250b8aad44bf3c512: Status 404 returned error can't find the container with id 4990bd6611f98307bf1feb2bc87117fe900f3e036a34780250b8aad44bf3c512 Jul 15 17:03:45.474374 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.474355 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:03:45.917525 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:45.917489 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" event={"ID":"29c867b6-c059-4d03-a672-5a5b3448f24a","Type":"ContainerStarted","Data":"4990bd6611f98307bf1feb2bc87117fe900f3e036a34780250b8aad44bf3c512"} Jul 15 17:03:48.933267 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:48.933235 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" event={"ID":"29c867b6-c059-4d03-a672-5a5b3448f24a","Type":"ContainerStarted","Data":"4181b9d29972154f579aaab6925d8ad6bc8841f3532c452354cb93ef86304eb2"} Jul 15 17:03:49.939001 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:49.938961 2566 generic.go:356] "Generic (PLEG): container finished" podID="29c867b6-c059-4d03-a672-5a5b3448f24a" containerID="4181b9d29972154f579aaab6925d8ad6bc8841f3532c452354cb93ef86304eb2" exitCode=0 Jul 15 17:03:49.939401 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:49.939022 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" event={"ID":"29c867b6-c059-4d03-a672-5a5b3448f24a","Type":"ContainerDied","Data":"4181b9d29972154f579aaab6925d8ad6bc8841f3532c452354cb93ef86304eb2"} Jul 15 17:03:50.946426 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:50.946390 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" event={"ID":"29c867b6-c059-4d03-a672-5a5b3448f24a","Type":"ContainerStarted","Data":"7e4c5f0a1697daca6ebb6c06c50a1d7b8ac6e5dcb98882fdb9881acd945aee99"} Jul 15 17:03:50.946903 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:50.946486 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:03:50.971578 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:03:50.971527 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" podStartSLOduration=2.555132076 podStartE2EDuration="6.971506349s" totalImagesPullingTime="3.360992571s" totalInitContainerRuntime="1.055381702s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:03:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:03:50.970816325 +0000 UTC m=+325.284521254" watchObservedRunningTime="2026-07-15 17:03:50.971506349 +0000 UTC m=+325.285211245" Jul 15 17:04:01.953565 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:01.953534 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fdf764586-98m57" Jul 15 17:04:04.659833 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.659798 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-667d575688-szp56"] Jul 15 17:04:04.670985 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.670959 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.679130 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.679112 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 17:04:04.679712 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.679698 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 17:04:04.686175 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.686152 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-667d575688-szp56"] Jul 15 17:04:04.756648 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.756615 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecdaec29-7580-4ff3-a9a9-b2f122f75ebc-tmp\") pod \"kube-auth-proxy-667d575688-szp56\" (UID: \"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc\") " pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.756648 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.756655 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdaec29-7580-4ff3-a9a9-b2f122f75ebc-tls-certs\") pod \"kube-auth-proxy-667d575688-szp56\" (UID: \"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc\") " pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.756858 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.756678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm4wz\" (UniqueName: \"kubernetes.io/projected/ecdaec29-7580-4ff3-a9a9-b2f122f75ebc-kube-api-access-sm4wz\") pod \"kube-auth-proxy-667d575688-szp56\" (UID: \"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc\") " pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.857502 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.857461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecdaec29-7580-4ff3-a9a9-b2f122f75ebc-tmp\") pod \"kube-auth-proxy-667d575688-szp56\" (UID: \"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc\") " pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.857502 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.857500 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdaec29-7580-4ff3-a9a9-b2f122f75ebc-tls-certs\") pod \"kube-auth-proxy-667d575688-szp56\" (UID: \"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc\") " pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.857777 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.857522 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm4wz\" (UniqueName: \"kubernetes.io/projected/ecdaec29-7580-4ff3-a9a9-b2f122f75ebc-kube-api-access-sm4wz\") pod \"kube-auth-proxy-667d575688-szp56\" (UID: \"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc\") " pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.859935 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.859908 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecdaec29-7580-4ff3-a9a9-b2f122f75ebc-tmp\") pod \"kube-auth-proxy-667d575688-szp56\" (UID: \"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc\") " pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.860053 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.860035 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ecdaec29-7580-4ff3-a9a9-b2f122f75ebc-tls-certs\") pod \"kube-auth-proxy-667d575688-szp56\" (UID: \"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc\") " pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.874028 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.873997 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm4wz\" (UniqueName: \"kubernetes.io/projected/ecdaec29-7580-4ff3-a9a9-b2f122f75ebc-kube-api-access-sm4wz\") pod \"kube-auth-proxy-667d575688-szp56\" (UID: \"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc\") " pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:04.979921 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:04.979835 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" Jul 15 17:04:05.048683 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.048651 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-w9q9k"] Jul 15 17:04:05.051249 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.051226 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:05.053881 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.053846 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-67qql\"" Jul 15 17:04:05.054090 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.054064 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 15 17:04:05.059215 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.059188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crtvf\" (UniqueName: \"kubernetes.io/projected/4cb130c2-cb80-4489-8830-7bc57842431b-kube-api-access-crtvf\") pod \"odh-model-controller-7998c67dc4-w9q9k\" (UID: \"4cb130c2-cb80-4489-8830-7bc57842431b\") " pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:05.059345 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.059237 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb130c2-cb80-4489-8830-7bc57842431b-cert\") pod \"odh-model-controller-7998c67dc4-w9q9k\" (UID: \"4cb130c2-cb80-4489-8830-7bc57842431b\") " pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:05.066989 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.066920 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-w9q9k"] Jul 15 17:04:05.140849 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.140799 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-667d575688-szp56"] Jul 15 17:04:05.143026 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:04:05.142994 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecdaec29_7580_4ff3_a9a9_b2f122f75ebc.slice/crio-54cb362ee100846da86a8d64e0a5daba8e148118b4e290edec5870e98f9b3500 WatchSource:0}: Error finding container 54cb362ee100846da86a8d64e0a5daba8e148118b4e290edec5870e98f9b3500: Status 404 returned error can't find the container with id 54cb362ee100846da86a8d64e0a5daba8e148118b4e290edec5870e98f9b3500 Jul 15 17:04:05.159670 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.159644 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crtvf\" (UniqueName: \"kubernetes.io/projected/4cb130c2-cb80-4489-8830-7bc57842431b-kube-api-access-crtvf\") pod \"odh-model-controller-7998c67dc4-w9q9k\" (UID: \"4cb130c2-cb80-4489-8830-7bc57842431b\") " pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:05.159852 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.159697 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb130c2-cb80-4489-8830-7bc57842431b-cert\") pod \"odh-model-controller-7998c67dc4-w9q9k\" (UID: \"4cb130c2-cb80-4489-8830-7bc57842431b\") " pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:05.159922 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:04:05.159852 2566 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 15 17:04:05.159974 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:04:05.159929 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb130c2-cb80-4489-8830-7bc57842431b-cert podName:4cb130c2-cb80-4489-8830-7bc57842431b nodeName:}" failed. No retries permitted until 2026-07-15 17:04:05.659906039 +0000 UTC m=+339.973610912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4cb130c2-cb80-4489-8830-7bc57842431b-cert") pod "odh-model-controller-7998c67dc4-w9q9k" (UID: "4cb130c2-cb80-4489-8830-7bc57842431b") : secret "odh-model-controller-webhook-cert" not found Jul 15 17:04:05.171056 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.171030 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crtvf\" (UniqueName: \"kubernetes.io/projected/4cb130c2-cb80-4489-8830-7bc57842431b-kube-api-access-crtvf\") pod \"odh-model-controller-7998c67dc4-w9q9k\" (UID: \"4cb130c2-cb80-4489-8830-7bc57842431b\") " pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:05.664014 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.663975 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb130c2-cb80-4489-8830-7bc57842431b-cert\") pod \"odh-model-controller-7998c67dc4-w9q9k\" (UID: \"4cb130c2-cb80-4489-8830-7bc57842431b\") " pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:05.666688 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.666662 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4cb130c2-cb80-4489-8830-7bc57842431b-cert\") pod \"odh-model-controller-7998c67dc4-w9q9k\" (UID: \"4cb130c2-cb80-4489-8830-7bc57842431b\") " pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:05.963449 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:05.963401 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:06.008582 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.008544 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fkr6t"] Jul 15 17:04:06.012299 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.012267 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:06.014151 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.014123 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" event={"ID":"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc","Type":"ContainerStarted","Data":"54cb362ee100846da86a8d64e0a5daba8e148118b4e290edec5870e98f9b3500"} Jul 15 17:04:06.015552 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.015533 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 17:04:06.015700 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.015583 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-stkgp\"" Jul 15 17:04:06.025846 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.025773 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fkr6t"] Jul 15 17:04:06.068151 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.068118 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbmtc\" (UniqueName: \"kubernetes.io/projected/d4b3c857-5833-43d7-8a0b-eedcccc6f10c-kube-api-access-sbmtc\") pod \"model-serving-api-54d89bd5fc-fkr6t\" (UID: \"d4b3c857-5833-43d7-8a0b-eedcccc6f10c\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:06.068319 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.068158 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4b3c857-5833-43d7-8a0b-eedcccc6f10c-tls-certs\") pod \"model-serving-api-54d89bd5fc-fkr6t\" (UID: \"d4b3c857-5833-43d7-8a0b-eedcccc6f10c\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:06.114583 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.114549 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-w9q9k"] Jul 15 17:04:06.118064 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:04:06.118033 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb130c2_cb80_4489_8830_7bc57842431b.slice/crio-c0d4cbdec518d0133348251d6f2d6f021db2e935524d96294a90501641e288be WatchSource:0}: Error finding container c0d4cbdec518d0133348251d6f2d6f021db2e935524d96294a90501641e288be: Status 404 returned error can't find the container with id c0d4cbdec518d0133348251d6f2d6f021db2e935524d96294a90501641e288be Jul 15 17:04:06.169209 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.169175 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbmtc\" (UniqueName: \"kubernetes.io/projected/d4b3c857-5833-43d7-8a0b-eedcccc6f10c-kube-api-access-sbmtc\") pod \"model-serving-api-54d89bd5fc-fkr6t\" (UID: \"d4b3c857-5833-43d7-8a0b-eedcccc6f10c\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:06.169209 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.169212 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4b3c857-5833-43d7-8a0b-eedcccc6f10c-tls-certs\") pod \"model-serving-api-54d89bd5fc-fkr6t\" (UID: \"d4b3c857-5833-43d7-8a0b-eedcccc6f10c\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:06.172255 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.172203 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d4b3c857-5833-43d7-8a0b-eedcccc6f10c-tls-certs\") pod \"model-serving-api-54d89bd5fc-fkr6t\" (UID: \"d4b3c857-5833-43d7-8a0b-eedcccc6f10c\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:06.182202 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.182171 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbmtc\" (UniqueName: \"kubernetes.io/projected/d4b3c857-5833-43d7-8a0b-eedcccc6f10c-kube-api-access-sbmtc\") pod \"model-serving-api-54d89bd5fc-fkr6t\" (UID: \"d4b3c857-5833-43d7-8a0b-eedcccc6f10c\") " pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:06.330434 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.330396 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:06.479268 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:06.479221 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fkr6t"] Jul 15 17:04:06.481427 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:04:06.481399 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b3c857_5833_43d7_8a0b_eedcccc6f10c.slice/crio-46733b6d3d2892e8be32073b32d14c4821640d0e7bde2e66449213043b9d4bbe WatchSource:0}: Error finding container 46733b6d3d2892e8be32073b32d14c4821640d0e7bde2e66449213043b9d4bbe: Status 404 returned error can't find the container with id 46733b6d3d2892e8be32073b32d14c4821640d0e7bde2e66449213043b9d4bbe Jul 15 17:04:07.020293 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:07.020246 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" event={"ID":"4cb130c2-cb80-4489-8830-7bc57842431b","Type":"ContainerStarted","Data":"c0d4cbdec518d0133348251d6f2d6f021db2e935524d96294a90501641e288be"} Jul 15 17:04:07.021625 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:07.021579 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" event={"ID":"d4b3c857-5833-43d7-8a0b-eedcccc6f10c","Type":"ContainerStarted","Data":"46733b6d3d2892e8be32073b32d14c4821640d0e7bde2e66449213043b9d4bbe"} Jul 15 17:04:10.102674 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.102635 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh"] Jul 15 17:04:10.106235 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.106214 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.108874 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.108850 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 17:04:10.109172 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.109145 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5j6f8\"" Jul 15 17:04:10.123773 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.123744 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh"] Jul 15 17:04:10.209450 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.209418 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.209580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.209457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.209580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.209476 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwnrr\" (UniqueName: \"kubernetes.io/projected/0739cb2b-1aec-41a8-ba72-bbc05e29579c-kube-api-access-cwnrr\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.209580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.209530 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.209580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.209577 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.209741 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.209617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.209741 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.209659 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.209741 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.209682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.209741 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.209710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310147 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310115 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310321 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310321 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310438 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310368 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310438 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310401 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310438 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310429 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310587 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310457 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwnrr\" (UniqueName: \"kubernetes.io/projected/0739cb2b-1aec-41a8-ba72-bbc05e29579c-kube-api-access-cwnrr\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310587 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310587 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310519 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310587 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310551 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.310807 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.310674 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.311035 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.311015 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.311239 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.311218 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.311330 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.311227 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.312959 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.312935 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.313178 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.313159 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.320773 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.320747 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0739cb2b-1aec-41a8-ba72-bbc05e29579c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.322883 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.322864 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwnrr\" (UniqueName: \"kubernetes.io/projected/0739cb2b-1aec-41a8-ba72-bbc05e29579c-kube-api-access-cwnrr\") pod \"data-science-gateway-data-science-gateway-class-7f964fb6786jxjh\" (UID: \"0739cb2b-1aec-41a8-ba72-bbc05e29579c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.419117 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.419022 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:10.585335 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:10.585141 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh"] Jul 15 17:04:10.587446 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:04:10.587422 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0739cb2b_1aec_41a8_ba72_bbc05e29579c.slice/crio-ad3c98bae5d97feafb291326a8f33333367e30b224a603519004a3d420ccc295 WatchSource:0}: Error finding container ad3c98bae5d97feafb291326a8f33333367e30b224a603519004a3d420ccc295: Status 404 returned error can't find the container with id ad3c98bae5d97feafb291326a8f33333367e30b224a603519004a3d420ccc295 Jul 15 17:04:11.049585 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:11.049543 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" event={"ID":"4cb130c2-cb80-4489-8830-7bc57842431b","Type":"ContainerStarted","Data":"a5a70982f8b732966ae77515d012c6add260424811b1289a3ef6448357c654f3"} Jul 15 17:04:11.049779 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:11.049666 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:11.051447 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:11.051415 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" event={"ID":"ecdaec29-7580-4ff3-a9a9-b2f122f75ebc","Type":"ContainerStarted","Data":"4f62b1fb15e4ca0ead98c5e2954f52a8af798a66c89c76f103e2958995e1eeee"} Jul 15 17:04:11.052998 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:11.052972 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" event={"ID":"d4b3c857-5833-43d7-8a0b-eedcccc6f10c","Type":"ContainerStarted","Data":"42cae5cce2e43b6e98e4b7929d12728444104703659c051240529c4e7d00dded"} Jul 15 17:04:11.053113 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:11.053093 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:11.054102 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:11.054083 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" event={"ID":"0739cb2b-1aec-41a8-ba72-bbc05e29579c","Type":"ContainerStarted","Data":"ad3c98bae5d97feafb291326a8f33333367e30b224a603519004a3d420ccc295"} Jul 15 17:04:11.074370 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:11.074328 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" podStartSLOduration=2.049560209 podStartE2EDuration="6.074313791s" totalImagesPullingTime="4.024753582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:04:11.073669348 +0000 UTC m=+345.387374246" watchObservedRunningTime="2026-07-15 17:04:11.074313791 +0000 UTC m=+345.388018680" Jul 15 17:04:11.096922 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:11.096867 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-667d575688-szp56" podStartSLOduration=2.109268867 podStartE2EDuration="7.096850194s" totalImagesPullingTime="4.987581327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:04:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:04:11.095195603 +0000 UTC m=+345.408900499" watchObservedRunningTime="2026-07-15 17:04:11.096850194 +0000 UTC m=+345.410555084" Jul 15 17:04:11.118052 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:11.117989 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" podStartSLOduration=2.467330884 podStartE2EDuration="6.117973521s" totalImagesPullingTime="3.650642637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:04:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:04:11.116804481 +0000 UTC m=+345.430509372" watchObservedRunningTime="2026-07-15 17:04:11.117973521 +0000 UTC m=+345.431678411" Jul 15 17:04:13.124904 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.124871 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:04:13.125150 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.124949 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:04:13.125150 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.124988 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:04:13.482890 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.482814 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9"] Jul 15 17:04:13.486380 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.486363 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.489330 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.489310 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 17:04:13.489459 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.489311 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 17:04:13.489459 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.489387 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kp5jp\"" Jul 15 17:04:13.489745 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.489733 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 17:04:13.502689 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.502664 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9"] Jul 15 17:04:13.537698 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.537664 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e112279e-c309-4462-945b-a91fc9faa1b3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.537698 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.537702 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4jcx\" (UniqueName: \"kubernetes.io/projected/e112279e-c309-4462-945b-a91fc9faa1b3-kube-api-access-k4jcx\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.537891 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.537782 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e112279e-c309-4462-945b-a91fc9faa1b3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.537891 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.537821 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e112279e-c309-4462-945b-a91fc9faa1b3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.537891 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.537857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e112279e-c309-4462-945b-a91fc9faa1b3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.537891 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.537870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e112279e-c309-4462-945b-a91fc9faa1b3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.538006 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.537926 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e112279e-c309-4462-945b-a91fc9faa1b3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.639824 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.639786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e112279e-c309-4462-945b-a91fc9faa1b3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.640008 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.639872 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e112279e-c309-4462-945b-a91fc9faa1b3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.640008 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.639934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e112279e-c309-4462-945b-a91fc9faa1b3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.640008 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.639957 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e112279e-c309-4462-945b-a91fc9faa1b3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.640008 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.639984 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e112279e-c309-4462-945b-a91fc9faa1b3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.640205 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.640030 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e112279e-c309-4462-945b-a91fc9faa1b3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.640205 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.640069 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4jcx\" (UniqueName: \"kubernetes.io/projected/e112279e-c309-4462-945b-a91fc9faa1b3-kube-api-access-k4jcx\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.641525 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.641435 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e112279e-c309-4462-945b-a91fc9faa1b3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.643174 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.643129 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e112279e-c309-4462-945b-a91fc9faa1b3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.643435 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.643401 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e112279e-c309-4462-945b-a91fc9faa1b3-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.643507 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.643441 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e112279e-c309-4462-945b-a91fc9faa1b3-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.643596 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.643579 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e112279e-c309-4462-945b-a91fc9faa1b3-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.649445 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.649413 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e112279e-c309-4462-945b-a91fc9faa1b3-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.649925 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.649898 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4jcx\" (UniqueName: \"kubernetes.io/projected/e112279e-c309-4462-945b-a91fc9faa1b3-kube-api-access-k4jcx\") pod \"istiod-openshift-gateway-94bb8fbfd-f6lt9\" (UID: \"e112279e-c309-4462-945b-a91fc9faa1b3\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.795845 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.795758 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:13.938966 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:13.938942 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9"] Jul 15 17:04:13.941129 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:04:13.941104 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode112279e_c309_4462_945b_a91fc9faa1b3.slice/crio-5994cb07a0b78c4b30ca8cff4ee254b0965d7c4dce59118297a23ef796589a27 WatchSource:0}: Error finding container 5994cb07a0b78c4b30ca8cff4ee254b0965d7c4dce59118297a23ef796589a27: Status 404 returned error can't find the container with id 5994cb07a0b78c4b30ca8cff4ee254b0965d7c4dce59118297a23ef796589a27 Jul 15 17:04:14.070181 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:14.070097 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" event={"ID":"0739cb2b-1aec-41a8-ba72-bbc05e29579c","Type":"ContainerStarted","Data":"17eaad6856f56faf1b3cf57c76e6b3e7e1089cf5b51c69609975aa3047a51b49"} Jul 15 17:04:14.071407 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:14.071377 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" event={"ID":"e112279e-c309-4462-945b-a91fc9faa1b3","Type":"ContainerStarted","Data":"5994cb07a0b78c4b30ca8cff4ee254b0965d7c4dce59118297a23ef796589a27"} Jul 15 17:04:14.090301 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:14.090257 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" podStartSLOduration=1.555379878 podStartE2EDuration="4.090241713s" totalImagesPullingTime="2.534861835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:04:14.089461906 +0000 UTC m=+348.403166808" watchObservedRunningTime="2026-07-15 17:04:14.090241713 +0000 UTC m=+348.403946668" Jul 15 17:04:14.419887 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:14.419851 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:14.425136 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:14.425099 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:15.077279 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:15.077241 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:15.078275 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:15.078250 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb6786jxjh" Jul 15 17:04:16.163892 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.163856 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:04:16.164198 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.163921 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:04:16.218583 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.218556 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f"] Jul 15 17:04:16.221820 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.221803 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:04:16.231994 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.231974 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 15 17:04:16.233058 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.233041 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-stwz9\"" Jul 15 17:04:16.246202 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.246178 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f"] Jul 15 17:04:16.261948 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.261902 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8c94165-df4d-4ece-8321-dbc69b172736-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ddh8f\" (UID: \"f8c94165-df4d-4ece-8321-dbc69b172736\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:04:16.262093 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.262011 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqlmr\" (UniqueName: \"kubernetes.io/projected/f8c94165-df4d-4ece-8321-dbc69b172736-kube-api-access-jqlmr\") pod \"llmisvc-controller-manager-777c6bd7d5-ddh8f\" (UID: \"f8c94165-df4d-4ece-8321-dbc69b172736\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:04:16.362664 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.362624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqlmr\" (UniqueName: \"kubernetes.io/projected/f8c94165-df4d-4ece-8321-dbc69b172736-kube-api-access-jqlmr\") pod \"llmisvc-controller-manager-777c6bd7d5-ddh8f\" (UID: \"f8c94165-df4d-4ece-8321-dbc69b172736\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:04:16.362867 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.362693 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8c94165-df4d-4ece-8321-dbc69b172736-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ddh8f\" (UID: \"f8c94165-df4d-4ece-8321-dbc69b172736\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:04:16.365152 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.365130 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f8c94165-df4d-4ece-8321-dbc69b172736-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ddh8f\" (UID: \"f8c94165-df4d-4ece-8321-dbc69b172736\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:04:16.373106 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.373082 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqlmr\" (UniqueName: \"kubernetes.io/projected/f8c94165-df4d-4ece-8321-dbc69b172736-kube-api-access-jqlmr\") pod \"llmisvc-controller-manager-777c6bd7d5-ddh8f\" (UID: \"f8c94165-df4d-4ece-8321-dbc69b172736\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:04:16.532907 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.532817 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:04:16.706980 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:16.706947 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f"] Jul 15 17:04:16.708997 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:04:16.708966 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf8c94165_df4d_4ece_8321_dbc69b172736.slice/crio-05cf92453c70083fcea0dabb762b7ad0b4a0f6b5c63ebb1ec5ff8bdb6f8d14d2 WatchSource:0}: Error finding container 05cf92453c70083fcea0dabb762b7ad0b4a0f6b5c63ebb1ec5ff8bdb6f8d14d2: Status 404 returned error can't find the container with id 05cf92453c70083fcea0dabb762b7ad0b4a0f6b5c63ebb1ec5ff8bdb6f8d14d2 Jul 15 17:04:17.089224 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:17.089168 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" event={"ID":"e112279e-c309-4462-945b-a91fc9faa1b3","Type":"ContainerStarted","Data":"2a8ca9539a5d3c03b22c71426c1368ecc38b8df46762435f71d14bef184225c3"} Jul 15 17:04:17.089426 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:17.089242 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:17.090347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:17.090322 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" event={"ID":"f8c94165-df4d-4ece-8321-dbc69b172736","Type":"ContainerStarted","Data":"05cf92453c70083fcea0dabb762b7ad0b4a0f6b5c63ebb1ec5ff8bdb6f8d14d2"} Jul 15 17:04:17.091144 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:17.091127 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" Jul 15 17:04:17.208155 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:17.208104 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-f6lt9" podStartSLOduration=1.987728319 podStartE2EDuration="4.208086421s" totalImagesPullingTime="2.220358102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:04:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:04:17.184094072 +0000 UTC m=+351.497798965" watchObservedRunningTime="2026-07-15 17:04:17.208086421 +0000 UTC m=+351.521791312" Jul 15 17:04:20.106305 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:20.106269 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" event={"ID":"f8c94165-df4d-4ece-8321-dbc69b172736","Type":"ContainerStarted","Data":"feb1d300aae13d7eaabea7b461fa96cd317ad2ef5ed8c253ba3357f91c7ab51f"} Jul 15 17:04:20.106669 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:20.106437 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:04:20.138809 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:20.138758 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" podStartSLOduration=1.163131575 podStartE2EDuration="4.138743629s" totalImagesPullingTime="2.975612054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:04:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:04:20.137294735 +0000 UTC m=+354.450999627" watchObservedRunningTime="2026-07-15 17:04:20.138743629 +0000 UTC m=+354.452448586" Jul 15 17:04:22.061887 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:22.061854 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-w9q9k" Jul 15 17:04:22.064214 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:22.064189 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fkr6t" Jul 15 17:04:51.113486 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:04:51.113455 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ddh8f" Jul 15 17:06:05.288120 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.288030 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz"] Jul 15 17:06:05.290960 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.290935 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:05.294336 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.294312 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 17:06:05.294336 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.294326 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 17:06:05.294534 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.294399 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9jw68\"" Jul 15 17:06:05.298562 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.298532 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917cbf1b-1c11-480a-9996-b42108bbb040-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jpfrz\" (UID: \"917cbf1b-1c11-480a-9996-b42108bbb040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:05.298798 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.298772 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2cjg\" (UniqueName: \"kubernetes.io/projected/917cbf1b-1c11-480a-9996-b42108bbb040-kube-api-access-b2cjg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jpfrz\" (UID: \"917cbf1b-1c11-480a-9996-b42108bbb040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:05.310947 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.310912 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz"] Jul 15 17:06:05.399461 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.399425 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2cjg\" (UniqueName: \"kubernetes.io/projected/917cbf1b-1c11-480a-9996-b42108bbb040-kube-api-access-b2cjg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jpfrz\" (UID: \"917cbf1b-1c11-480a-9996-b42108bbb040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:05.399682 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.399475 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917cbf1b-1c11-480a-9996-b42108bbb040-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jpfrz\" (UID: \"917cbf1b-1c11-480a-9996-b42108bbb040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:05.399856 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.399840 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917cbf1b-1c11-480a-9996-b42108bbb040-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jpfrz\" (UID: \"917cbf1b-1c11-480a-9996-b42108bbb040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:05.412998 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.412968 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2cjg\" (UniqueName: \"kubernetes.io/projected/917cbf1b-1c11-480a-9996-b42108bbb040-kube-api-access-b2cjg\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jpfrz\" (UID: \"917cbf1b-1c11-480a-9996-b42108bbb040\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:05.602929 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.602895 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:05.762691 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:06:05.762660 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod917cbf1b_1c11_480a_9996_b42108bbb040.slice/crio-482be3bdac458d479965687f108652e8e3cc6b011298ae414df7ef8ed95b5458 WatchSource:0}: Error finding container 482be3bdac458d479965687f108652e8e3cc6b011298ae414df7ef8ed95b5458: Status 404 returned error can't find the container with id 482be3bdac458d479965687f108652e8e3cc6b011298ae414df7ef8ed95b5458 Jul 15 17:06:05.765543 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:05.765520 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz"] Jul 15 17:06:06.586401 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:06.586360 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" event={"ID":"917cbf1b-1c11-480a-9996-b42108bbb040","Type":"ContainerStarted","Data":"482be3bdac458d479965687f108652e8e3cc6b011298ae414df7ef8ed95b5458"} Jul 15 17:06:11.616198 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:11.616164 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" event={"ID":"917cbf1b-1c11-480a-9996-b42108bbb040","Type":"ContainerStarted","Data":"7bda64cd612d9389bcf658fa3ba56f3a9199bbd796301fc68c9bb73dfbd4e0c9"} Jul 15 17:06:11.616715 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:11.616276 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:11.643586 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:11.643530 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" podStartSLOduration=1.20378739 podStartE2EDuration="6.643509853s" totalImagesPullingTime="5.439722463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:11.641932674 +0000 UTC m=+465.955637560" watchObservedRunningTime="2026-07-15 17:06:11.643509853 +0000 UTC m=+465.957214745" Jul 15 17:06:22.624248 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:22.624214 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:30.442703 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.442666 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz"] Jul 15 17:06:30.443148 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.442852 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" podUID="917cbf1b-1c11-480a-9996-b42108bbb040" containerName="manager" containerID="cri-o://7bda64cd612d9389bcf658fa3ba56f3a9199bbd796301fc68c9bb73dfbd4e0c9" gracePeriod=10 Jul 15 17:06:30.703721 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.703654 2566 generic.go:356] "Generic (PLEG): container finished" podID="917cbf1b-1c11-480a-9996-b42108bbb040" containerID="7bda64cd612d9389bcf658fa3ba56f3a9199bbd796301fc68c9bb73dfbd4e0c9" exitCode=0 Jul 15 17:06:30.703881 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.703723 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" event={"ID":"917cbf1b-1c11-480a-9996-b42108bbb040","Type":"ContainerDied","Data":"7bda64cd612d9389bcf658fa3ba56f3a9199bbd796301fc68c9bb73dfbd4e0c9"} Jul 15 17:06:30.703881 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.703772 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" event={"ID":"917cbf1b-1c11-480a-9996-b42108bbb040","Type":"ContainerDied","Data":"482be3bdac458d479965687f108652e8e3cc6b011298ae414df7ef8ed95b5458"} Jul 15 17:06:30.703881 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.703789 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="482be3bdac458d479965687f108652e8e3cc6b011298ae414df7ef8ed95b5458" Jul 15 17:06:30.705751 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.705735 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:30.826542 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.826516 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2cjg\" (UniqueName: \"kubernetes.io/projected/917cbf1b-1c11-480a-9996-b42108bbb040-kube-api-access-b2cjg\") pod \"917cbf1b-1c11-480a-9996-b42108bbb040\" (UID: \"917cbf1b-1c11-480a-9996-b42108bbb040\") " Jul 15 17:06:30.826763 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.826654 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917cbf1b-1c11-480a-9996-b42108bbb040-extensions-socket-volume\") pod \"917cbf1b-1c11-480a-9996-b42108bbb040\" (UID: \"917cbf1b-1c11-480a-9996-b42108bbb040\") " Jul 15 17:06:30.827077 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.827040 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/917cbf1b-1c11-480a-9996-b42108bbb040-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "917cbf1b-1c11-480a-9996-b42108bbb040" (UID: "917cbf1b-1c11-480a-9996-b42108bbb040"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:06:30.829238 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.829204 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/917cbf1b-1c11-480a-9996-b42108bbb040-kube-api-access-b2cjg" (OuterVolumeSpecName: "kube-api-access-b2cjg") pod "917cbf1b-1c11-480a-9996-b42108bbb040" (UID: "917cbf1b-1c11-480a-9996-b42108bbb040"). InnerVolumeSpecName "kube-api-access-b2cjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:06:30.927504 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.927469 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/917cbf1b-1c11-480a-9996-b42108bbb040-extensions-socket-volume\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:06:30.927504 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:30.927497 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2cjg\" (UniqueName: \"kubernetes.io/projected/917cbf1b-1c11-480a-9996-b42108bbb040-kube-api-access-b2cjg\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:06:31.708633 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:31.708585 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz" Jul 15 17:06:31.739378 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:31.739345 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz"] Jul 15 17:06:31.744285 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:31.744254 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jpfrz"] Jul 15 17:06:32.188236 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:32.188199 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="917cbf1b-1c11-480a-9996-b42108bbb040" path="/var/lib/kubelet/pods/917cbf1b-1c11-480a-9996-b42108bbb040/volumes" Jul 15 17:06:46.945949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:46.945911 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7"] Jul 15 17:06:46.946404 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:46.946255 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="917cbf1b-1c11-480a-9996-b42108bbb040" containerName="manager" Jul 15 17:06:46.946404 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:46.946267 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="917cbf1b-1c11-480a-9996-b42108bbb040" containerName="manager" Jul 15 17:06:46.946404 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:46.946338 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="917cbf1b-1c11-480a-9996-b42108bbb040" containerName="manager" Jul 15 17:06:46.950549 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:46.950529 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:46.953792 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:46.953772 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-w5qnl\"" Jul 15 17:06:46.969686 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:46.969663 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7"] Jul 15 17:06:47.073102 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.073064 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ab5193b-e598-4539-926d-007d5cec7ac5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.073102 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.073101 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.073347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.073128 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.073347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.073180 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.073347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.073227 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.073347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.073265 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.073347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.073294 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mvdv\" (UniqueName: \"kubernetes.io/projected/9ab5193b-e598-4539-926d-007d5cec7ac5-kube-api-access-5mvdv\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.073580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.073374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.073580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.073417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.174974 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.174940 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.174974 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.174977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175245 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ab5193b-e598-4539-926d-007d5cec7ac5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175245 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175039 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175245 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175080 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175245 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175109 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175245 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175141 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175245 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175245 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175199 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mvdv\" (UniqueName: \"kubernetes.io/projected/9ab5193b-e598-4539-926d-007d5cec7ac5-kube-api-access-5mvdv\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175628 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175449 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175628 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175574 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175753 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175670 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175753 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175744 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.175962 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.175930 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9ab5193b-e598-4539-926d-007d5cec7ac5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.177672 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.177647 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.177851 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.177833 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.208315 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.208245 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mvdv\" (UniqueName: \"kubernetes.io/projected/9ab5193b-e598-4539-926d-007d5cec7ac5-kube-api-access-5mvdv\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.212970 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.212946 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ab5193b-e598-4539-926d-007d5cec7ac5-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-bx4k7\" (UID: \"9ab5193b-e598-4539-926d-007d5cec7ac5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.261000 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.260971 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:47.418224 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.418187 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7"] Jul 15 17:06:47.421114 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:06:47.421085 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab5193b_e598_4539_926d_007d5cec7ac5.slice/crio-17bc34599d6ba76136d674650b748698949239a1867358abcbe0a907a30bc5f5 WatchSource:0}: Error finding container 17bc34599d6ba76136d674650b748698949239a1867358abcbe0a907a30bc5f5: Status 404 returned error can't find the container with id 17bc34599d6ba76136d674650b748698949239a1867358abcbe0a907a30bc5f5 Jul 15 17:06:47.423128 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.423094 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:06:47.423216 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.423169 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:06:47.423216 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.423206 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:06:47.784074 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.783973 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" event={"ID":"9ab5193b-e598-4539-926d-007d5cec7ac5","Type":"ContainerStarted","Data":"a7acdf34f4da30215092db715b54f6369a4d739ad38116979566df8ea03791ab"} Jul 15 17:06:47.784074 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.784031 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" event={"ID":"9ab5193b-e598-4539-926d-007d5cec7ac5","Type":"ContainerStarted","Data":"17bc34599d6ba76136d674650b748698949239a1867358abcbe0a907a30bc5f5"} Jul 15 17:06:47.808737 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:47.808675 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" podStartSLOduration=1.8086523429999999 podStartE2EDuration="1.808652343s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:47.805352113 +0000 UTC m=+502.119057006" watchObservedRunningTime="2026-07-15 17:06:47.808652343 +0000 UTC m=+502.122357235" Jul 15 17:06:48.262249 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:48.262211 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:48.266909 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:48.266882 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:48.792633 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:48.791825 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:48.795828 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:48.795801 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:06:50.203954 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.203922 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:06:50.207559 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.207540 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:06:50.210349 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.210329 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 17:06:50.210477 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.210357 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 17:06:50.211737 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.211718 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 17:06:50.211813 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.211763 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7clnc\"" Jul 15 17:06:50.218243 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.218219 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:06:50.261092 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.261047 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:06:50.306771 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.306735 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5w4p\" (UniqueName: \"kubernetes.io/projected/75bc0d06-0fa6-40f9-9c85-e5a399d8b232-kube-api-access-x5w4p\") pod \"limitador-limitador-54d6d988d5-z8fvg\" (UID: \"75bc0d06-0fa6-40f9-9c85-e5a399d8b232\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:06:50.306960 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.306830 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/75bc0d06-0fa6-40f9-9c85-e5a399d8b232-config-file\") pod \"limitador-limitador-54d6d988d5-z8fvg\" (UID: \"75bc0d06-0fa6-40f9-9c85-e5a399d8b232\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:06:50.408157 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.408123 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/75bc0d06-0fa6-40f9-9c85-e5a399d8b232-config-file\") pod \"limitador-limitador-54d6d988d5-z8fvg\" (UID: \"75bc0d06-0fa6-40f9-9c85-e5a399d8b232\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:06:50.408361 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.408195 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5w4p\" (UniqueName: \"kubernetes.io/projected/75bc0d06-0fa6-40f9-9c85-e5a399d8b232-kube-api-access-x5w4p\") pod \"limitador-limitador-54d6d988d5-z8fvg\" (UID: \"75bc0d06-0fa6-40f9-9c85-e5a399d8b232\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:06:50.408830 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.408799 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/75bc0d06-0fa6-40f9-9c85-e5a399d8b232-config-file\") pod \"limitador-limitador-54d6d988d5-z8fvg\" (UID: \"75bc0d06-0fa6-40f9-9c85-e5a399d8b232\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:06:50.417658 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.417588 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5w4p\" (UniqueName: \"kubernetes.io/projected/75bc0d06-0fa6-40f9-9c85-e5a399d8b232-kube-api-access-x5w4p\") pod \"limitador-limitador-54d6d988d5-z8fvg\" (UID: \"75bc0d06-0fa6-40f9-9c85-e5a399d8b232\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:06:50.519006 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.518921 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:06:50.662110 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.662083 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:06:50.664555 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:06:50.664528 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75bc0d06_0fa6_40f9_9c85_e5a399d8b232.slice/crio-c5392a8656a55ff69def9273ab34bd8f2317c267fa25595f079ca76d562251b0 WatchSource:0}: Error finding container c5392a8656a55ff69def9273ab34bd8f2317c267fa25595f079ca76d562251b0: Status 404 returned error can't find the container with id c5392a8656a55ff69def9273ab34bd8f2317c267fa25595f079ca76d562251b0 Jul 15 17:06:50.801014 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:50.800926 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" event={"ID":"75bc0d06-0fa6-40f9-9c85-e5a399d8b232","Type":"ContainerStarted","Data":"c5392a8656a55ff69def9273ab34bd8f2317c267fa25595f079ca76d562251b0"} Jul 15 17:06:54.822705 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:54.822667 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" event={"ID":"75bc0d06-0fa6-40f9-9c85-e5a399d8b232","Type":"ContainerStarted","Data":"6a595cc53c253debbb8aa0bfbf819ff97ec5159953171a68cd5f3611760fd251"} Jul 15 17:06:54.823112 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:54.822826 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:06:54.849063 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:06:54.845787 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" podStartSLOduration=1.4720524799999999 podStartE2EDuration="4.845766263s" totalImagesPullingTime="3.373713783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:06:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:06:54.84165941 +0000 UTC m=+509.155364300" watchObservedRunningTime="2026-07-15 17:06:54.845766263 +0000 UTC m=+509.159471155" Jul 15 17:07:05.829324 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:07:05.829296 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-z8fvg" Jul 15 17:08:08.392367 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:08.392329 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:08:16.215090 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:16.215060 2566 generic.go:356] "Generic (PLEG): container finished" podID="9ab5193b-e598-4539-926d-007d5cec7ac5" containerID="a7acdf34f4da30215092db715b54f6369a4d739ad38116979566df8ea03791ab" exitCode=0 Jul 15 17:08:16.215521 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:16.215104 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" event={"ID":"9ab5193b-e598-4539-926d-007d5cec7ac5","Type":"ContainerDied","Data":"a7acdf34f4da30215092db715b54f6369a4d739ad38116979566df8ea03791ab"} Jul 15 17:08:16.215521 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:16.215510 2566 scope.go:117] "RemoveContainer" containerID="a7acdf34f4da30215092db715b54f6369a4d739ad38116979566df8ea03791ab" Jul 15 17:08:16.216525 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:16.216497 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:08:16.216579 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:16.216566 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:08:16.216638 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:16.216624 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:08:17.221290 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:17.221256 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" event={"ID":"9ab5193b-e598-4539-926d-007d5cec7ac5","Type":"ContainerStarted","Data":"b223eb691750008368613b95a06aa740de00456fce9aa8bd338231404492a00b"} Jul 15 17:08:17.261933 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:17.261462 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:08:17.262801 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:17.262763 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-bx4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 15 17:08:17.262943 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:17.262840 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" podUID="9ab5193b-e598-4539-926d-007d5cec7ac5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 15 17:08:18.262461 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:18.262417 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-bx4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 15 17:08:18.263100 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:18.263056 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" podUID="9ab5193b-e598-4539-926d-007d5cec7ac5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 15 17:08:18.792398 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:18.792357 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:08:19.262937 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:19.262899 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-bx4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 15 17:08:19.263553 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:19.263522 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" podUID="9ab5193b-e598-4539-926d-007d5cec7ac5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 15 17:08:20.262398 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:20.262358 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-bx4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 15 17:08:20.262747 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:20.262712 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" podUID="9ab5193b-e598-4539-926d-007d5cec7ac5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 15 17:08:21.265661 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:21.265631 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:08:21.266679 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:21.266655 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-bx4k7" Jul 15 17:08:26.123855 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:26.123825 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:08:26.126808 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:26.126786 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:08:26.130231 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:26.130211 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:08:26.133662 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:26.133636 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:08:28.925538 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:28.925506 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:08:34.696553 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:08:34.696516 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:09:06.695054 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:06.695017 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:09:16.391852 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:16.391814 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:09:31.833589 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.833549 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k"] Jul 15 17:09:31.836262 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.836243 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:31.840447 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.840423 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4bf9z\"" Jul 15 17:09:31.841338 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.841320 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 15 17:09:31.841338 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.841333 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 17:09:31.841503 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.841374 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 17:09:31.848243 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.848218 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k"] Jul 15 17:09:31.915128 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.915088 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:31.915128 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.915123 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:31.915344 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.915148 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:31.915344 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.915192 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:31.915344 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.915285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:31.915444 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:31.915356 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2x6r\" (UniqueName: \"kubernetes.io/projected/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-kube-api-access-v2x6r\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.016425 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.016386 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2x6r\" (UniqueName: \"kubernetes.io/projected/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-kube-api-access-v2x6r\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.016636 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.016450 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.016636 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.016568 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.016636 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.016624 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.016823 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.016736 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.016880 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.016853 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.016933 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.016873 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.016984 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.016924 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.017141 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.017123 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.019705 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.019667 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.019970 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.019948 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.026252 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.026224 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2x6r\" (UniqueName: \"kubernetes.io/projected/3d3372e5-1cb5-4e7a-8d6b-a7670561cb84-kube-api-access-v2x6r\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-hck2k\" (UID: \"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.146966 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.146884 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:32.291713 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.291646 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k"] Jul 15 17:09:32.294105 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:09:32.294078 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3372e5_1cb5_4e7a_8d6b_a7670561cb84.slice/crio-1d1f8708f3935dd7131655500f77960c0af61960fab8f5a33d7d69f883208335 WatchSource:0}: Error finding container 1d1f8708f3935dd7131655500f77960c0af61960fab8f5a33d7d69f883208335: Status 404 returned error can't find the container with id 1d1f8708f3935dd7131655500f77960c0af61960fab8f5a33d7d69f883208335 Jul 15 17:09:32.295489 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.295468 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 17:09:32.583157 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:32.583118 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" event={"ID":"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84","Type":"ContainerStarted","Data":"1d1f8708f3935dd7131655500f77960c0af61960fab8f5a33d7d69f883208335"} Jul 15 17:09:33.589711 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:33.589674 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" event={"ID":"3d3372e5-1cb5-4e7a-8d6b-a7670561cb84","Type":"ContainerStarted","Data":"dbdae19f01371660246f7b19964ca313078f7845f1781bf53325cae5ef2479c6"} Jul 15 17:09:33.611076 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:33.610976 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" podStartSLOduration=1.5462010849999999 podStartE2EDuration="2.610959461s" totalImagesPullingTime="1.064758376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:09:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:09:33.609344711 +0000 UTC m=+667.923049605" watchObservedRunningTime="2026-07-15 17:09:33.610959461 +0000 UTC m=+667.924664352" Jul 15 17:09:34.014188 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:34.014097 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:09:42.147850 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:42.147805 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:42.148379 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:42.147882 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:42.160925 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:42.160887 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:42.644303 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:42.644264 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-hck2k" Jul 15 17:09:51.906876 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:09:51.906835 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:12:00.095665 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:12:00.095558 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:12:08.890499 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:12:08.890457 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:12:21.000135 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:12:21.000085 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:12:26.184827 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:12:26.184794 2566 scope.go:117] "RemoveContainer" containerID="7bda64cd612d9389bcf658fa3ba56f3a9199bbd796301fc68c9bb73dfbd4e0c9" Jul 15 17:12:29.501852 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:12:29.501807 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:12:40.598959 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:12:40.598922 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:13:26.155003 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:13:26.154974 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:13:26.155478 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:13:26.155284 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:13:26.161812 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:13:26.161790 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:13:26.167631 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:13:26.162539 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:13:51.500436 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:13:51.500399 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:14:00.697387 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:14:00.697354 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:14:10.802214 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:14:10.802177 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:14:16.501349 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:14:16.501310 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:14:27.611225 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:14:27.611192 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:14:37.426180 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:14:37.426146 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:14:47.412485 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:14:47.412399 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:14:57.606338 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:14:57.606200 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:15:22.010411 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:15:22.010378 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:15:33.103528 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:15:33.103491 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:15:58.897108 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:15:58.897071 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:16:10.907539 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:16:10.907501 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:16:27.902648 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:16:27.902597 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:16:37.001156 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:16:37.001119 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:16:46.597821 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:16:46.597784 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:16:55.800349 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:16:55.800312 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:17:04.502183 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:17:04.502149 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:18:26.191517 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:18:26.191490 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:18:26.194170 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:18:26.193350 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:18:26.198530 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:18:26.198505 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:18:26.200277 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:18:26.200256 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:18:52.694113 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:18:52.694067 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:19:02.699089 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:19:02.699049 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:19:13.010245 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:19:13.010202 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:19:25.416370 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:19:25.416334 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:19:35.185169 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:19:35.185134 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:19:46.197661 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:19:46.197624 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:19:53.996862 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:19:53.996822 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:20:05.108409 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:20:05.108375 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:20:17.126206 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:20:17.126166 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:20:26.220377 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:20:26.220340 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:20:38.220054 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:20:38.220016 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:21:01.392739 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:21:01.392641 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:21:08.553582 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:21:08.553541 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:21:18.907105 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:21:18.907064 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:21:28.006884 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:21:28.006848 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:21:34.231614 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:21:34.231575 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:21:52.009291 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:21:52.009236 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:22:01.404126 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:22:01.404085 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:22:10.414248 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:22:10.414208 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:22:21.612652 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:22:21.612597 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:22:31.031955 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:22:31.031914 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:22:40.435383 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:22:40.435345 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:22:52.308187 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:22:52.308143 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:23:02.598246 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:02.598211 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:23:11.909616 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:11.909570 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:23:22.301984 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:22.301950 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:23:26.220078 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:26.219958 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:23:26.224271 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:26.224253 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:23:26.226646 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:26.226625 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:23:26.230314 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:26.230300 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:23:34.704277 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:34.704237 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:23:44.724024 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:44.723987 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:23:57.190346 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:23:57.190265 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:24:08.489107 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:08.489067 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:24:17.889095 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:17.889051 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:24:29.787766 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:29.787728 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:24:46.099693 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:46.099653 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:24:50.451418 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.451373 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz"] Jul 15 17:24:50.455376 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.455348 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.458228 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.458207 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-554abb3e-openshift-default-dockercfg-gns7x\"" Jul 15 17:24:50.479244 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.479206 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz"] Jul 15 17:24:50.577912 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.577875 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-envoy\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.578084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.577918 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a498064-8e25-43af-a89d-b717fc957287-istio-podinfo\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.578084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.577949 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-socket\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.578084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.578026 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-credential-socket\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.578084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.578058 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-certs\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.578084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.578078 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sclx\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-kube-api-access-6sclx\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.578265 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.578102 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a498064-8e25-43af-a89d-b717fc957287-istiod-ca-cert\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.578265 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.578143 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-data\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.578265 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.578159 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-istio-token\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.678704 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.678664 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-envoy\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.678704 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.678710 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a498064-8e25-43af-a89d-b717fc957287-istio-podinfo\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.678949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.678732 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-socket\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.678949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.678765 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-credential-socket\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.678949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.678783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-certs\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.678949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.678802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sclx\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-kube-api-access-6sclx\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.678949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.678822 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a498064-8e25-43af-a89d-b717fc957287-istiod-ca-cert\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.678949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.678862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-data\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.678949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.678887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-istio-token\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.679318 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.679263 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-socket\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.679395 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.679372 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-credential-socket\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.679469 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.679451 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-data\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.679630 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.679591 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-certs\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.680074 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.680051 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a498064-8e25-43af-a89d-b717fc957287-istiod-ca-cert\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.681170 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.681144 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-envoy\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.681417 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.681399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a498064-8e25-43af-a89d-b717fc957287-istio-podinfo\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.687967 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.687940 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-istio-token\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.688065 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.688048 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sclx\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-kube-api-access-6sclx\") pod \"e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.768956 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.768863 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:50.915746 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.915717 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz"] Jul 15 17:24:50.918269 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:24:50.918240 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a498064_8e25_43af_a89d_b717fc957287.slice/crio-32f50058b84edbdef79e625531305f4c7b478a03e1b4df8e9b87b38bc86b939b WatchSource:0}: Error finding container 32f50058b84edbdef79e625531305f4c7b478a03e1b4df8e9b87b38bc86b939b: Status 404 returned error can't find the container with id 32f50058b84edbdef79e625531305f4c7b478a03e1b4df8e9b87b38bc86b939b Jul 15 17:24:50.920511 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.920481 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:24:50.920597 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.920549 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:24:50.920597 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:50.920586 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:24:51.004365 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:51.004322 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" event={"ID":"1a498064-8e25-43af-a89d-b717fc957287","Type":"ContainerStarted","Data":"4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781"} Jul 15 17:24:51.004520 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:51.004376 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" event={"ID":"1a498064-8e25-43af-a89d-b717fc957287","Type":"ContainerStarted","Data":"32f50058b84edbdef79e625531305f4c7b478a03e1b4df8e9b87b38bc86b939b"} Jul 15 17:24:51.028123 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:51.028019 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" podStartSLOduration=1.028003679 podStartE2EDuration="1.028003679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:24:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:24:51.027194894 +0000 UTC m=+1585.340899787" watchObservedRunningTime="2026-07-15 17:24:51.028003679 +0000 UTC m=+1585.341708573" Jul 15 17:24:51.769940 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:51.769895 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:51.771207 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:51.771180 2566 patch_prober.go:28] interesting pod/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 15 17:24:51.771301 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:51.771234 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 15 17:24:52.773903 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:52.773871 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:53.014564 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:53.014529 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:53.015696 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:53.015677 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:24:58.686811 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:24:58.686775 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:25:07.094142 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:07.094109 2566 generic.go:356] "Generic (PLEG): container finished" podID="1a498064-8e25-43af-a89d-b717fc957287" containerID="4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781" exitCode=0 Jul 15 17:25:07.094651 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:07.094178 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" event={"ID":"1a498064-8e25-43af-a89d-b717fc957287","Type":"ContainerDied","Data":"4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781"} Jul 15 17:25:07.094651 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:07.094559 2566 scope.go:117] "RemoveContainer" containerID="4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781" Jul 15 17:25:07.095670 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:07.095638 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:25:07.095734 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:07.095711 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:25:07.095772 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:07.095744 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:25:08.015198 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:08.015153 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:25:08.100619 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:08.100574 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" event={"ID":"1a498064-8e25-43af-a89d-b717fc957287","Type":"ContainerStarted","Data":"f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30"} Jul 15 17:25:08.769503 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:08.769463 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:25:08.770529 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:08.770492 2566 patch_prober.go:28] interesting pod/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 15 17:25:08.770663 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:08.770557 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 15 17:25:09.770019 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:09.769977 2566 patch_prober.go:28] interesting pod/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 15 17:25:09.770672 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:09.770633 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 15 17:25:10.770356 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:10.770317 2566 patch_prober.go:28] interesting pod/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 15 17:25:10.770971 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:10.770936 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 15 17:25:11.769908 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:11.769871 2566 patch_prober.go:28] interesting pod/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 15 17:25:11.770228 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:11.770187 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 15 17:25:12.773443 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:12.773414 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:25:12.773922 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:12.773846 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:25:12.774696 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:25:12.774673 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:26:00.192847 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:00.192810 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:26:12.033730 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.033686 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz"] Jul 15 17:26:12.034308 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.033952 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" containerID="cri-o://f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30" gracePeriod=30 Jul 15 17:26:12.590515 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.590482 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2"] Jul 15 17:26:12.593736 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.593712 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.597057 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.597037 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-a2500ffb-gw-openshift-default-dockercfg-kng9n\"" Jul 15 17:26:12.613417 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.613387 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2"] Jul 15 17:26:12.649171 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.649138 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-credential-socket\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.649347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.649271 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-podinfo\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.649347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.649316 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-socket\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.649347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.649336 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m884p\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-kube-api-access-m884p\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.649480 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.649355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-certs\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.649480 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.649386 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-data\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.649480 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.649435 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-envoy\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.649592 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.649477 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ab27805-7aab-45c3-9645-8e0279d20bb7-istiod-ca-cert\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.649592 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.649501 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-token\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.750721 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.750677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ab27805-7aab-45c3-9645-8e0279d20bb7-istiod-ca-cert\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.750895 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.750739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-token\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.750895 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.750768 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-credential-socket\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.750895 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.750858 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-podinfo\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.750895 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.750886 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-socket\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.751170 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.750908 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m884p\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-kube-api-access-m884p\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.751170 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.750939 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-certs\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.751170 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.750975 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-data\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.751170 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.751020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-envoy\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.751385 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.751306 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-socket\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.751385 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.751353 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-credential-socket\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.751474 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.751386 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-certs\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.751517 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.751478 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-data\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.751674 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.751650 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ab27805-7aab-45c3-9645-8e0279d20bb7-istiod-ca-cert\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.753422 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.753404 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-envoy\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.753666 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.753646 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-podinfo\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.760991 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.760964 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-token\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.761261 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.761238 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m884p\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-kube-api-access-m884p\") pod \"e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:12.775891 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.775863 2566 patch_prober.go:28] interesting pod/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 17:26:12.776038 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.775927 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 17:26:12.905729 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:12.905632 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:13.057590 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:13.057563 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2"] Jul 15 17:26:13.060065 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:26:13.060038 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ab27805_7aab_45c3_9645_8e0279d20bb7.slice/crio-66b1959b7da47970559c2c6bd471b181800cf7b479b9078f211a9f686937df67 WatchSource:0}: Error finding container 66b1959b7da47970559c2c6bd471b181800cf7b479b9078f211a9f686937df67: Status 404 returned error can't find the container with id 66b1959b7da47970559c2c6bd471b181800cf7b479b9078f211a9f686937df67 Jul 15 17:26:13.062080 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:13.062048 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:26:13.062160 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:13.062112 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:26:13.062160 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:13.062140 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:26:13.418696 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:13.418661 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" event={"ID":"1ab27805-7aab-45c3-9645-8e0279d20bb7","Type":"ContainerStarted","Data":"81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277"} Jul 15 17:26:13.418894 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:13.418714 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" event={"ID":"1ab27805-7aab-45c3-9645-8e0279d20bb7","Type":"ContainerStarted","Data":"66b1959b7da47970559c2c6bd471b181800cf7b479b9078f211a9f686937df67"} Jul 15 17:26:13.447390 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:13.447335 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" podStartSLOduration=1.447319499 podStartE2EDuration="1.447319499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:26:13.446270064 +0000 UTC m=+1667.759974957" watchObservedRunningTime="2026-07-15 17:26:13.447319499 +0000 UTC m=+1667.761024391" Jul 15 17:26:13.906485 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:13.906447 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:14.912255 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:14.912226 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:15.428677 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:15.428640 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:15.429731 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:15.429714 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:17.294121 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.294059 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:26:17.399023 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.398991 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-envoy\") pod \"1a498064-8e25-43af-a89d-b717fc957287\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " Jul 15 17:26:17.399175 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399032 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-istio-token\") pod \"1a498064-8e25-43af-a89d-b717fc957287\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " Jul 15 17:26:17.399175 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399081 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a498064-8e25-43af-a89d-b717fc957287-istiod-ca-cert\") pod \"1a498064-8e25-43af-a89d-b717fc957287\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " Jul 15 17:26:17.399175 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399116 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-certs\") pod \"1a498064-8e25-43af-a89d-b717fc957287\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " Jul 15 17:26:17.399175 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399141 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sclx\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-kube-api-access-6sclx\") pod \"1a498064-8e25-43af-a89d-b717fc957287\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " Jul 15 17:26:17.399379 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399197 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-credential-socket\") pod \"1a498064-8e25-43af-a89d-b717fc957287\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " Jul 15 17:26:17.399379 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399252 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-socket\") pod \"1a498064-8e25-43af-a89d-b717fc957287\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " Jul 15 17:26:17.399379 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399296 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a498064-8e25-43af-a89d-b717fc957287-istio-podinfo\") pod \"1a498064-8e25-43af-a89d-b717fc957287\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " Jul 15 17:26:17.399379 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399324 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-data\") pod \"1a498064-8e25-43af-a89d-b717fc957287\" (UID: \"1a498064-8e25-43af-a89d-b717fc957287\") " Jul 15 17:26:17.399576 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399475 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a498064-8e25-43af-a89d-b717fc957287" (UID: "1a498064-8e25-43af-a89d-b717fc957287"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:26:17.399576 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399530 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a498064-8e25-43af-a89d-b717fc957287-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a498064-8e25-43af-a89d-b717fc957287" (UID: "1a498064-8e25-43af-a89d-b717fc957287"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:26:17.399727 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399685 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a498064-8e25-43af-a89d-b717fc957287" (UID: "1a498064-8e25-43af-a89d-b717fc957287"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:26:17.399789 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399772 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a498064-8e25-43af-a89d-b717fc957287" (UID: "1a498064-8e25-43af-a89d-b717fc957287"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:26:17.399852 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399837 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a498064-8e25-43af-a89d-b717fc957287-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:26:17.399904 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399855 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:26:17.399904 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399868 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:26:17.399904 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.399882 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:26:17.400576 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.400548 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a498064-8e25-43af-a89d-b717fc957287" (UID: "1a498064-8e25-43af-a89d-b717fc957287"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:26:17.401730 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.401698 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a498064-8e25-43af-a89d-b717fc957287" (UID: "1a498064-8e25-43af-a89d-b717fc957287"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:26:17.401830 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.401789 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a498064-8e25-43af-a89d-b717fc957287-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a498064-8e25-43af-a89d-b717fc957287" (UID: "1a498064-8e25-43af-a89d-b717fc957287"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:26:17.401936 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.401921 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a498064-8e25-43af-a89d-b717fc957287" (UID: "1a498064-8e25-43af-a89d-b717fc957287"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:26:17.401979 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.401951 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-kube-api-access-6sclx" (OuterVolumeSpecName: "kube-api-access-6sclx") pod "1a498064-8e25-43af-a89d-b717fc957287" (UID: "1a498064-8e25-43af-a89d-b717fc957287"). InnerVolumeSpecName "kube-api-access-6sclx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:26:17.445468 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.445429 2566 generic.go:356] "Generic (PLEG): container finished" podID="1a498064-8e25-43af-a89d-b717fc957287" containerID="f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30" exitCode=0 Jul 15 17:26:17.445686 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.445498 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" Jul 15 17:26:17.445686 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.445515 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" event={"ID":"1a498064-8e25-43af-a89d-b717fc957287","Type":"ContainerDied","Data":"f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30"} Jul 15 17:26:17.445686 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.445592 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz" event={"ID":"1a498064-8e25-43af-a89d-b717fc957287","Type":"ContainerDied","Data":"32f50058b84edbdef79e625531305f4c7b478a03e1b4df8e9b87b38bc86b939b"} Jul 15 17:26:17.445686 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.445628 2566 scope.go:117] "RemoveContainer" containerID="f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30" Jul 15 17:26:17.457945 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.457922 2566 scope.go:117] "RemoveContainer" containerID="4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781" Jul 15 17:26:17.466510 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.466488 2566 scope.go:117] "RemoveContainer" containerID="f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30" Jul 15 17:26:17.467014 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:26:17.466989 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30\": container with ID starting with f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30 not found: ID does not exist" containerID="f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30" Jul 15 17:26:17.467101 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.467021 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30"} err="failed to get container status \"f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30\": rpc error: code = NotFound desc = could not find container \"f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30\": container with ID starting with f2f2a2df5060171321f2d7f7857eb24dcc3ad8e1249aa0d23321c2dc88a62c30 not found: ID does not exist" Jul 15 17:26:17.467101 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.467039 2566 scope.go:117] "RemoveContainer" containerID="4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781" Jul 15 17:26:17.467368 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:26:17.467351 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781\": container with ID starting with 4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781 not found: ID does not exist" containerID="4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781" Jul 15 17:26:17.467409 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.467371 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781"} err="failed to get container status \"4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781\": rpc error: code = NotFound desc = could not find container \"4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781\": container with ID starting with 4d868df1dea57c6840cf0bb02b8801aa6396cc4e1703891041f830b7c50dd781 not found: ID does not exist" Jul 15 17:26:17.468469 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.468446 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz"] Jul 15 17:26:17.472203 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.472171 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-554abb3e-openshift-default-6bfcb659c-244vz"] Jul 15 17:26:17.500574 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.500539 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sclx\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-kube-api-access-6sclx\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:26:17.500574 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.500571 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a498064-8e25-43af-a89d-b717fc957287-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:26:17.500781 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.500587 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:26:17.500781 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.500629 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a498064-8e25-43af-a89d-b717fc957287-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:26:17.500781 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:17.500644 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a498064-8e25-43af-a89d-b717fc957287-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:26:18.187863 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:18.187830 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a498064-8e25-43af-a89d-b717fc957287" path="/var/lib/kubelet/pods/1a498064-8e25-43af-a89d-b717fc957287/volumes" Jul 15 17:26:21.884364 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:21.884329 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:26:29.513966 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:29.513930 2566 generic.go:356] "Generic (PLEG): container finished" podID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerID="81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277" exitCode=0 Jul 15 17:26:29.514358 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:29.513983 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" event={"ID":"1ab27805-7aab-45c3-9645-8e0279d20bb7","Type":"ContainerDied","Data":"81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277"} Jul 15 17:26:29.514591 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:29.514393 2566 scope.go:117] "RemoveContainer" containerID="81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277" Jul 15 17:26:29.515426 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:29.515396 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:26:29.515482 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:29.515469 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:26:29.515519 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:29.515502 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:26:30.429148 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:30.429111 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:30.520494 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:30.520461 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" event={"ID":"1ab27805-7aab-45c3-9645-8e0279d20bb7","Type":"ContainerStarted","Data":"e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581"} Jul 15 17:26:30.906685 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:30.906639 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:30.907951 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:30.907928 2566 patch_prober.go:28] interesting pod/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 15 17:26:30.908036 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:30.907979 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 15 17:26:31.906496 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:31.906443 2566 patch_prober.go:28] interesting pod/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 15 17:26:31.906929 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:31.906518 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 15 17:26:32.906904 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:32.906871 2566 patch_prober.go:28] interesting pod/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 15 17:26:32.907327 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:32.906924 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 15 17:26:33.906868 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:33.906832 2566 patch_prober.go:28] interesting pod/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 15 17:26:33.907054 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:33.906887 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 15 17:26:34.909882 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:34.909852 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:34.910281 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:34.910122 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:26:34.910893 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:26:34.910872 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:27:18.468919 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:18.468838 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2"] Jul 15 17:27:18.469429 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:18.469042 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" containerID="cri-o://e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581" gracePeriod=30 Jul 15 17:27:19.912213 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:19.912171 2566 patch_prober.go:28] interesting pod/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 17:27:19.912598 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:19.912227 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 17:27:21.200697 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:21.200666 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:27:23.724765 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.724738 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:27:23.771983 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.771890 2566 generic.go:356] "Generic (PLEG): container finished" podID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerID="e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581" exitCode=0 Jul 15 17:27:23.771983 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.771939 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" Jul 15 17:27:23.772218 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.771970 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" event={"ID":"1ab27805-7aab-45c3-9645-8e0279d20bb7","Type":"ContainerDied","Data":"e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581"} Jul 15 17:27:23.772218 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.772040 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2" event={"ID":"1ab27805-7aab-45c3-9645-8e0279d20bb7","Type":"ContainerDied","Data":"66b1959b7da47970559c2c6bd471b181800cf7b479b9078f211a9f686937df67"} Jul 15 17:27:23.772218 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.772071 2566 scope.go:117] "RemoveContainer" containerID="e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581" Jul 15 17:27:23.781073 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.781052 2566 scope.go:117] "RemoveContainer" containerID="81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277" Jul 15 17:27:23.789803 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.789779 2566 scope.go:117] "RemoveContainer" containerID="e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581" Jul 15 17:27:23.790259 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:27:23.790197 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581\": container with ID starting with e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581 not found: ID does not exist" containerID="e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581" Jul 15 17:27:23.790259 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.790231 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581"} err="failed to get container status \"e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581\": rpc error: code = NotFound desc = could not find container \"e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581\": container with ID starting with e57a8f72c8e59425bc87320850dc5ed2cb350f9fd34f80be7b79abc976c49581 not found: ID does not exist" Jul 15 17:27:23.790259 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.790252 2566 scope.go:117] "RemoveContainer" containerID="81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277" Jul 15 17:27:23.790569 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:27:23.790536 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277\": container with ID starting with 81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277 not found: ID does not exist" containerID="81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277" Jul 15 17:27:23.790801 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.790579 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277"} err="failed to get container status \"81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277\": rpc error: code = NotFound desc = could not find container \"81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277\": container with ID starting with 81fdc30415e76ddb943ce5be579e7777ac2a3ed9d1994a134dc620a3bb082277 not found: ID does not exist" Jul 15 17:27:23.808408 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808368 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-token\") pod \"1ab27805-7aab-45c3-9645-8e0279d20bb7\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " Jul 15 17:27:23.808562 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808474 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-socket\") pod \"1ab27805-7aab-45c3-9645-8e0279d20bb7\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " Jul 15 17:27:23.808562 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808492 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-data\") pod \"1ab27805-7aab-45c3-9645-8e0279d20bb7\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " Jul 15 17:27:23.808562 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808521 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-credential-socket\") pod \"1ab27805-7aab-45c3-9645-8e0279d20bb7\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " Jul 15 17:27:23.808562 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808538 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ab27805-7aab-45c3-9645-8e0279d20bb7-istiod-ca-cert\") pod \"1ab27805-7aab-45c3-9645-8e0279d20bb7\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " Jul 15 17:27:23.808801 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808564 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-podinfo\") pod \"1ab27805-7aab-45c3-9645-8e0279d20bb7\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " Jul 15 17:27:23.808801 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808638 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m884p\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-kube-api-access-m884p\") pod \"1ab27805-7aab-45c3-9645-8e0279d20bb7\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " Jul 15 17:27:23.808801 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808665 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-certs\") pod \"1ab27805-7aab-45c3-9645-8e0279d20bb7\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " Jul 15 17:27:23.808801 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808685 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-envoy\") pod \"1ab27805-7aab-45c3-9645-8e0279d20bb7\" (UID: \"1ab27805-7aab-45c3-9645-8e0279d20bb7\") " Jul 15 17:27:23.809005 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.808816 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1ab27805-7aab-45c3-9645-8e0279d20bb7" (UID: "1ab27805-7aab-45c3-9645-8e0279d20bb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:27:23.809557 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.809442 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1ab27805-7aab-45c3-9645-8e0279d20bb7" (UID: "1ab27805-7aab-45c3-9645-8e0279d20bb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:27:23.809557 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.809519 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ab27805-7aab-45c3-9645-8e0279d20bb7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1ab27805-7aab-45c3-9645-8e0279d20bb7" (UID: "1ab27805-7aab-45c3-9645-8e0279d20bb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:27:23.809741 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.809579 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-data" (OuterVolumeSpecName: "istio-data") pod "1ab27805-7aab-45c3-9645-8e0279d20bb7" (UID: "1ab27805-7aab-45c3-9645-8e0279d20bb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:27:23.809922 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.809894 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1ab27805-7aab-45c3-9645-8e0279d20bb7" (UID: "1ab27805-7aab-45c3-9645-8e0279d20bb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:27:23.811769 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.810989 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:27:23.812160 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.812129 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-token" (OuterVolumeSpecName: "istio-token") pod "1ab27805-7aab-45c3-9645-8e0279d20bb7" (UID: "1ab27805-7aab-45c3-9645-8e0279d20bb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:27:23.813365 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.813340 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1ab27805-7aab-45c3-9645-8e0279d20bb7" (UID: "1ab27805-7aab-45c3-9645-8e0279d20bb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:27:23.813494 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.813409 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1ab27805-7aab-45c3-9645-8e0279d20bb7" (UID: "1ab27805-7aab-45c3-9645-8e0279d20bb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:27:23.818563 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.818539 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-kube-api-access-m884p" (OuterVolumeSpecName: "kube-api-access-m884p") pod "1ab27805-7aab-45c3-9645-8e0279d20bb7" (UID: "1ab27805-7aab-45c3-9645-8e0279d20bb7"). InnerVolumeSpecName "kube-api-access-m884p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:27:23.911484 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.911443 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:27:23.911484 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.911478 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:27:23.911484 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.911487 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1ab27805-7aab-45c3-9645-8e0279d20bb7-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:27:23.911484 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.911495 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:27:23.911798 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.911504 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m884p\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-kube-api-access-m884p\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:27:23.911798 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.911513 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:27:23.911798 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.911520 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:27:23.911798 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:23.911528 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1ab27805-7aab-45c3-9645-8e0279d20bb7-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:27:24.094242 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.094208 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2"] Jul 15 17:27:24.099053 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.099025 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-a2500ffb-gw-openshift-default-796897df9-vlcj2"] Jul 15 17:27:24.188123 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.188088 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" path="/var/lib/kubelet/pods/1ab27805-7aab-45c3-9645-8e0279d20bb7/volumes" Jul 15 17:27:24.951508 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.951465 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k"] Jul 15 17:27:24.952050 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952024 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" Jul 15 17:27:24.952050 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952041 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" Jul 15 17:27:24.952165 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952052 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" Jul 15 17:27:24.952165 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952061 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" Jul 15 17:27:24.952165 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952092 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" Jul 15 17:27:24.952165 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952102 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" Jul 15 17:27:24.952165 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952116 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" Jul 15 17:27:24.952165 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952125 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" Jul 15 17:27:24.952472 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952215 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" Jul 15 17:27:24.952472 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952227 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" Jul 15 17:27:24.952472 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952238 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a498064-8e25-43af-a89d-b717fc957287" containerName="istio-proxy" Jul 15 17:27:24.952472 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.952247 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ab27805-7aab-45c3-9645-8e0279d20bb7" containerName="istio-proxy" Jul 15 17:27:24.957045 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.957021 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:24.959695 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.959594 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-297e6a4a-openshift-default-dockercfg-m69gq\"" Jul 15 17:27:24.967050 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:24.967023 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k"] Jul 15 17:27:25.121388 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.121350 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-data\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.121388 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.121389 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b50c53-0ced-4625-b3e8-4911fe2702fe-istiod-ca-cert\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.121701 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.121408 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-envoy\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.121701 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.121507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-token\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.121701 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.121590 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-credential-socket\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.121701 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.121692 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-podinfo\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.121866 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.121724 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-socket\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.121866 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.121742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlpqk\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-kube-api-access-wlpqk\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.121866 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.121767 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-certs\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223007 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.222910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-credential-socket\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223007 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.222960 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-podinfo\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223007 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.222977 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-socket\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223291 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlpqk\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-kube-api-access-wlpqk\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223291 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223167 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-certs\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223291 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223218 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-data\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223291 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b50c53-0ced-4625-b3e8-4911fe2702fe-istiod-ca-cert\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223291 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223283 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-envoy\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223654 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223361 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-token\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223654 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223447 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-socket\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223654 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223527 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-certs\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223654 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223584 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-data\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.223654 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.223647 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-credential-socket\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.224192 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.224159 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b50c53-0ced-4625-b3e8-4911fe2702fe-istiod-ca-cert\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.226422 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.226399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-envoy\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.226547 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.226529 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-podinfo\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.237207 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.237179 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlpqk\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-kube-api-access-wlpqk\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.238415 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.238357 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-token\") pod \"e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.269124 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.269093 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:25.418638 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.418586 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k"] Jul 15 17:27:25.420978 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:27:25.420949 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b50c53_0ced_4625_b3e8_4911fe2702fe.slice/crio-78c9fb79b5980bcf2cc97c6072df9c4a0d818c3d578ddbedd7b9645de3f42c24 WatchSource:0}: Error finding container 78c9fb79b5980bcf2cc97c6072df9c4a0d818c3d578ddbedd7b9645de3f42c24: Status 404 returned error can't find the container with id 78c9fb79b5980bcf2cc97c6072df9c4a0d818c3d578ddbedd7b9645de3f42c24 Jul 15 17:27:25.423082 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.423052 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:27:25.423147 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.423117 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:27:25.423186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.423155 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:27:25.786647 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.786532 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" event={"ID":"91b50c53-0ced-4625-b3e8-4911fe2702fe","Type":"ContainerStarted","Data":"cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08"} Jul 15 17:27:25.786647 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.786626 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" event={"ID":"91b50c53-0ced-4625-b3e8-4911fe2702fe","Type":"ContainerStarted","Data":"78c9fb79b5980bcf2cc97c6072df9c4a0d818c3d578ddbedd7b9645de3f42c24"} Jul 15 17:27:25.810380 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:25.810325 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" podStartSLOduration=1.810309391 podStartE2EDuration="1.810309391s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:27:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:27:25.808374374 +0000 UTC m=+1740.122079268" watchObservedRunningTime="2026-07-15 17:27:25.810309391 +0000 UTC m=+1740.124014282" Jul 15 17:27:26.269780 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:26.269736 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:26.275167 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:26.275146 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:26.792436 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:26.792404 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:26.793360 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:26.793342 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:33.890985 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:33.890952 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:27:42.414926 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:42.414341 2566 patch_prober.go:28] interesting pod/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:50440->10.132.0.42:15021: read: connection reset by peer" start-of-body= Jul 15 17:27:42.414926 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:42.414424 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:50440->10.132.0.42:15021: read: connection reset by peer" Jul 15 17:27:42.886067 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:42.886034 2566 generic.go:356] "Generic (PLEG): container finished" podID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerID="cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08" exitCode=0 Jul 15 17:27:42.886237 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:42.886097 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" event={"ID":"91b50c53-0ced-4625-b3e8-4911fe2702fe","Type":"ContainerDied","Data":"cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08"} Jul 15 17:27:42.886485 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:42.886473 2566 scope.go:117] "RemoveContainer" containerID="cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08" Jul 15 17:27:42.887493 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:42.887461 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:27:42.887562 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:42.887536 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:27:42.887619 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:42.887568 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:27:43.892723 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:43.892685 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" event={"ID":"91b50c53-0ced-4625-b3e8-4911fe2702fe","Type":"ContainerStarted","Data":"aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f"} Jul 15 17:27:44.270065 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:44.269950 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:44.270260 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:44.270094 2566 patch_prober.go:28] interesting pod/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 15 17:27:44.270260 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:44.270138 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 15 17:27:45.270203 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:45.270152 2566 patch_prober.go:28] interesting pod/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 15 17:27:45.270760 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:45.270226 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 15 17:27:46.269930 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:46.269888 2566 patch_prober.go:28] interesting pod/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 15 17:27:46.270263 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:46.270232 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 15 17:27:47.269769 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:47.269724 2566 patch_prober.go:28] interesting pod/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 15 17:27:47.269983 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:47.269793 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 15 17:27:48.272853 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:48.272824 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:48.273220 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:48.273104 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:27:48.273936 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:27:48.273918 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:28:26.248646 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:26.248589 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:28:26.254149 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:26.254123 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:28:26.254719 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:26.254699 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:28:26.260859 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:26.260837 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:28:44.490776 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:44.490740 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:28:57.376046 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.376010 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k"] Jul 15 17:28:57.376621 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.376226 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" containerID="cri-o://aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f" gracePeriod=30 Jul 15 17:28:57.673171 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.673071 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2"] Jul 15 17:28:57.676752 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.676730 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.679987 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.679964 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6feb2776-openshift-default-dockercfg-szk4r\"" Jul 15 17:28:57.689317 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.689294 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2"] Jul 15 17:28:57.690545 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.690520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-podinfo\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.690699 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.690557 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhtlx\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-kube-api-access-fhtlx\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.690699 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.690585 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e111df2-b999-4ab4-bb58-e998cb5379ef-istiod-ca-cert\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.690699 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.690682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-credential-socket\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.690928 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.690710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-envoy\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.690928 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.690731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-token\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.690928 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.690800 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-socket\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.690928 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.690847 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-certs\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.690928 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.690870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-data\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.791676 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.791637 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-certs\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.791676 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.791677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-data\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.791917 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.791730 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-podinfo\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.791917 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.791761 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhtlx\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-kube-api-access-fhtlx\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.791917 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.791798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e111df2-b999-4ab4-bb58-e998cb5379ef-istiod-ca-cert\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.791917 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.791849 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-credential-socket\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.791917 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.791881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-envoy\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.791917 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.791905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-token\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.792180 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.791970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-socket\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.792180 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.792103 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-certs\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.792440 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.792416 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-data\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.792595 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.792476 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-credential-socket\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.792595 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.792541 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-socket\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.792922 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.792901 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e111df2-b999-4ab4-bb58-e998cb5379ef-istiod-ca-cert\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.794870 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.794844 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-envoy\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.795184 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.795162 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-podinfo\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.801942 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.801910 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhtlx\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-kube-api-access-fhtlx\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.802098 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.802075 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-token\") pod \"e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:57.988471 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:57.988371 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:58.132689 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:58.132664 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2"] Jul 15 17:28:58.134912 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:28:58.134881 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e111df2_b999_4ab4_bb58_e998cb5379ef.slice/crio-025c82707c68f9b28ee43539636b0b218af3f88f2d3e0e924e2bbc868e728fbb WatchSource:0}: Error finding container 025c82707c68f9b28ee43539636b0b218af3f88f2d3e0e924e2bbc868e728fbb: Status 404 returned error can't find the container with id 025c82707c68f9b28ee43539636b0b218af3f88f2d3e0e924e2bbc868e728fbb Jul 15 17:28:58.137112 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:58.137077 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:28:58.137201 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:58.137145 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:28:58.137201 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:58.137171 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:28:58.252373 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:58.252272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" event={"ID":"6e111df2-b999-4ab4-bb58-e998cb5379ef","Type":"ContainerStarted","Data":"efc99279d5c33c0536d04b0fdd1d02fd5fd42e20a7dbd7ed487b1f0089c66501"} Jul 15 17:28:58.252373 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:58.252341 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" event={"ID":"6e111df2-b999-4ab4-bb58-e998cb5379ef","Type":"ContainerStarted","Data":"025c82707c68f9b28ee43539636b0b218af3f88f2d3e0e924e2bbc868e728fbb"} Jul 15 17:28:58.273943 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:58.273892 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" podStartSLOduration=1.273876753 podStartE2EDuration="1.273876753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:28:58.272439972 +0000 UTC m=+1832.586144867" watchObservedRunningTime="2026-07-15 17:28:58.273876753 +0000 UTC m=+1832.587581644" Jul 15 17:28:58.989262 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:58.989225 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:28:59.994172 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:28:59.994142 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:29:00.265094 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:00.265007 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:29:00.266111 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:00.266093 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:29:02.626992 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.626956 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:29:02.738577 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.738547 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-envoy\") pod \"91b50c53-0ced-4625-b3e8-4911fe2702fe\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " Jul 15 17:29:02.738784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.738593 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-podinfo\") pod \"91b50c53-0ced-4625-b3e8-4911fe2702fe\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " Jul 15 17:29:02.738784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.738639 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b50c53-0ced-4625-b3e8-4911fe2702fe-istiod-ca-cert\") pod \"91b50c53-0ced-4625-b3e8-4911fe2702fe\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " Jul 15 17:29:02.738784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.738684 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-certs\") pod \"91b50c53-0ced-4625-b3e8-4911fe2702fe\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " Jul 15 17:29:02.738951 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.738830 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlpqk\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-kube-api-access-wlpqk\") pod \"91b50c53-0ced-4625-b3e8-4911fe2702fe\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " Jul 15 17:29:02.738951 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.738876 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-token\") pod \"91b50c53-0ced-4625-b3e8-4911fe2702fe\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " Jul 15 17:29:02.739058 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.738951 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-data\") pod \"91b50c53-0ced-4625-b3e8-4911fe2702fe\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " Jul 15 17:29:02.739058 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.738975 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-credential-socket\") pod \"91b50c53-0ced-4625-b3e8-4911fe2702fe\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " Jul 15 17:29:02.739058 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.738993 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "91b50c53-0ced-4625-b3e8-4911fe2702fe" (UID: "91b50c53-0ced-4625-b3e8-4911fe2702fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:29:02.739058 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.739007 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-socket\") pod \"91b50c53-0ced-4625-b3e8-4911fe2702fe\" (UID: \"91b50c53-0ced-4625-b3e8-4911fe2702fe\") " Jul 15 17:29:02.739269 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.739069 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91b50c53-0ced-4625-b3e8-4911fe2702fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "91b50c53-0ced-4625-b3e8-4911fe2702fe" (UID: "91b50c53-0ced-4625-b3e8-4911fe2702fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:29:02.739269 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.739194 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "91b50c53-0ced-4625-b3e8-4911fe2702fe" (UID: "91b50c53-0ced-4625-b3e8-4911fe2702fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:29:02.739525 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.739395 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:29:02.739525 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.739419 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91b50c53-0ced-4625-b3e8-4911fe2702fe-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:29:02.739525 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.739435 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:29:02.739525 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.739578 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "91b50c53-0ced-4625-b3e8-4911fe2702fe" (UID: "91b50c53-0ced-4625-b3e8-4911fe2702fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:29:02.740050 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.740023 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "91b50c53-0ced-4625-b3e8-4911fe2702fe" (UID: "91b50c53-0ced-4625-b3e8-4911fe2702fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:29:02.740994 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.740967 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "91b50c53-0ced-4625-b3e8-4911fe2702fe" (UID: "91b50c53-0ced-4625-b3e8-4911fe2702fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:29:02.741100 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.740971 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "91b50c53-0ced-4625-b3e8-4911fe2702fe" (UID: "91b50c53-0ced-4625-b3e8-4911fe2702fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:29:02.741155 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.741090 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-kube-api-access-wlpqk" (OuterVolumeSpecName: "kube-api-access-wlpqk") pod "91b50c53-0ced-4625-b3e8-4911fe2702fe" (UID: "91b50c53-0ced-4625-b3e8-4911fe2702fe"). InnerVolumeSpecName "kube-api-access-wlpqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:29:02.741218 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.741185 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "91b50c53-0ced-4625-b3e8-4911fe2702fe" (UID: "91b50c53-0ced-4625-b3e8-4911fe2702fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:29:02.840720 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.840686 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlpqk\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-kube-api-access-wlpqk\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:29:02.840720 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.840717 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:29:02.840720 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.840726 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:29:02.840964 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.840734 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:29:02.840964 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.840743 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:29:02.840964 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:02.840751 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91b50c53-0ced-4625-b3e8-4911fe2702fe-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:29:03.280267 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.280224 2566 generic.go:356] "Generic (PLEG): container finished" podID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerID="aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f" exitCode=0 Jul 15 17:29:03.280440 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.280275 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" event={"ID":"91b50c53-0ced-4625-b3e8-4911fe2702fe","Type":"ContainerDied","Data":"aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f"} Jul 15 17:29:03.280440 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.280297 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" Jul 15 17:29:03.280440 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.280328 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k" event={"ID":"91b50c53-0ced-4625-b3e8-4911fe2702fe","Type":"ContainerDied","Data":"78c9fb79b5980bcf2cc97c6072df9c4a0d818c3d578ddbedd7b9645de3f42c24"} Jul 15 17:29:03.280440 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.280358 2566 scope.go:117] "RemoveContainer" containerID="aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f" Jul 15 17:29:03.290044 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.290025 2566 scope.go:117] "RemoveContainer" containerID="cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08" Jul 15 17:29:03.298815 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.298791 2566 scope.go:117] "RemoveContainer" containerID="aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f" Jul 15 17:29:03.299085 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:29:03.299065 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f\": container with ID starting with aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f not found: ID does not exist" containerID="aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f" Jul 15 17:29:03.299141 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.299095 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f"} err="failed to get container status \"aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f\": rpc error: code = NotFound desc = could not find container \"aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f\": container with ID starting with aa0f6e965fd25a9c0f32276de7e7f4c5980768643f000f737cd908df67bd447f not found: ID does not exist" Jul 15 17:29:03.299141 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.299112 2566 scope.go:117] "RemoveContainer" containerID="cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08" Jul 15 17:29:03.299400 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:29:03.299381 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08\": container with ID starting with cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08 not found: ID does not exist" containerID="cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08" Jul 15 17:29:03.299438 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.299407 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08"} err="failed to get container status \"cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08\": rpc error: code = NotFound desc = could not find container \"cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08\": container with ID starting with cc8a893ca046e6deb9d285f20246bb70e761da904128e3918993f4ed3c574c08 not found: ID does not exist" Jul 15 17:29:03.305135 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.305108 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k"] Jul 15 17:29:03.310937 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:03.310913 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-297e6a4a-openshift-default-7dbbd8bcdf-clj7k"] Jul 15 17:29:04.187723 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:04.187686 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" path="/var/lib/kubelet/pods/91b50c53-0ced-4625-b3e8-4911fe2702fe/volumes" Jul 15 17:29:06.188526 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:06.188494 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:29:14.335913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:14.335874 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerID="efc99279d5c33c0536d04b0fdd1d02fd5fd42e20a7dbd7ed487b1f0089c66501" exitCode=0 Jul 15 17:29:14.336388 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:14.335945 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" event={"ID":"6e111df2-b999-4ab4-bb58-e998cb5379ef","Type":"ContainerDied","Data":"efc99279d5c33c0536d04b0fdd1d02fd5fd42e20a7dbd7ed487b1f0089c66501"} Jul 15 17:29:14.336388 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:14.336338 2566 scope.go:117] "RemoveContainer" containerID="efc99279d5c33c0536d04b0fdd1d02fd5fd42e20a7dbd7ed487b1f0089c66501" Jul 15 17:29:14.337342 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:14.337312 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:29:14.337417 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:14.337399 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:29:14.337469 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:14.337455 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:29:15.265994 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:15.265936 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:29:15.342857 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:15.342825 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" event={"ID":"6e111df2-b999-4ab4-bb58-e998cb5379ef","Type":"ContainerStarted","Data":"122e5194414204d3607072547454a7050ce8a22997bd88337cd9e015437e5932"} Jul 15 17:29:15.988865 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:15.988823 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:29:15.990068 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:15.990040 2566 patch_prober.go:28] interesting pod/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 15 17:29:15.990199 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:15.990090 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 15 17:29:16.989434 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:16.989397 2566 patch_prober.go:28] interesting pod/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 15 17:29:16.989868 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:16.989455 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 15 17:29:17.989042 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:17.988999 2566 patch_prober.go:28] interesting pod/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 15 17:29:17.989229 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:17.989073 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 15 17:29:18.988949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:18.988911 2566 patch_prober.go:28] interesting pod/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 15 17:29:18.989357 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:18.988968 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 15 17:29:19.993233 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:19.993200 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:29:19.993659 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:19.993526 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:29:19.994193 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:29:19.994178 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:30:16.589534 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:16.589449 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:30:28.528286 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:28.528246 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2"] Jul 15 17:30:28.528763 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:28.528479 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" containerID="cri-o://122e5194414204d3607072547454a7050ce8a22997bd88337cd9e015437e5932" gracePeriod=30 Jul 15 17:30:33.726580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.726550 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerID="122e5194414204d3607072547454a7050ce8a22997bd88337cd9e015437e5932" exitCode=0 Jul 15 17:30:33.726977 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.726643 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" event={"ID":"6e111df2-b999-4ab4-bb58-e998cb5379ef","Type":"ContainerDied","Data":"122e5194414204d3607072547454a7050ce8a22997bd88337cd9e015437e5932"} Jul 15 17:30:33.726977 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.726727 2566 scope.go:117] "RemoveContainer" containerID="efc99279d5c33c0536d04b0fdd1d02fd5fd42e20a7dbd7ed487b1f0089c66501" Jul 15 17:30:33.786152 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.786126 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:30:33.901923 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.901826 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e111df2-b999-4ab4-bb58-e998cb5379ef-istiod-ca-cert\") pod \"6e111df2-b999-4ab4-bb58-e998cb5379ef\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " Jul 15 17:30:33.902095 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.901987 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-envoy\") pod \"6e111df2-b999-4ab4-bb58-e998cb5379ef\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " Jul 15 17:30:33.902095 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902028 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhtlx\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-kube-api-access-fhtlx\") pod \"6e111df2-b999-4ab4-bb58-e998cb5379ef\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " Jul 15 17:30:33.902095 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902081 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-credential-socket\") pod \"6e111df2-b999-4ab4-bb58-e998cb5379ef\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " Jul 15 17:30:33.902273 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902103 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-podinfo\") pod \"6e111df2-b999-4ab4-bb58-e998cb5379ef\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " Jul 15 17:30:33.902273 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902112 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e111df2-b999-4ab4-bb58-e998cb5379ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e111df2-b999-4ab4-bb58-e998cb5379ef" (UID: "6e111df2-b999-4ab4-bb58-e998cb5379ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:30:33.902273 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902123 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-socket\") pod \"6e111df2-b999-4ab4-bb58-e998cb5379ef\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " Jul 15 17:30:33.902273 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902160 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-token\") pod \"6e111df2-b999-4ab4-bb58-e998cb5379ef\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " Jul 15 17:30:33.902273 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902203 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-data\") pod \"6e111df2-b999-4ab4-bb58-e998cb5379ef\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " Jul 15 17:30:33.902273 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902225 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-certs\") pod \"6e111df2-b999-4ab4-bb58-e998cb5379ef\" (UID: \"6e111df2-b999-4ab4-bb58-e998cb5379ef\") " Jul 15 17:30:33.902567 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902439 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e111df2-b999-4ab4-bb58-e998cb5379ef" (UID: "6e111df2-b999-4ab4-bb58-e998cb5379ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:30:33.902658 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902625 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:30:33.902658 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902646 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e111df2-b999-4ab4-bb58-e998cb5379ef-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:30:33.902998 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902658 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e111df2-b999-4ab4-bb58-e998cb5379ef" (UID: "6e111df2-b999-4ab4-bb58-e998cb5379ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:30:33.902998 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.902725 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e111df2-b999-4ab4-bb58-e998cb5379ef" (UID: "6e111df2-b999-4ab4-bb58-e998cb5379ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:30:33.903105 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.903019 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e111df2-b999-4ab4-bb58-e998cb5379ef" (UID: "6e111df2-b999-4ab4-bb58-e998cb5379ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:30:33.904486 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.904461 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e111df2-b999-4ab4-bb58-e998cb5379ef" (UID: "6e111df2-b999-4ab4-bb58-e998cb5379ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:30:33.904664 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.904549 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e111df2-b999-4ab4-bb58-e998cb5379ef" (UID: "6e111df2-b999-4ab4-bb58-e998cb5379ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:30:33.904867 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.904843 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-kube-api-access-fhtlx" (OuterVolumeSpecName: "kube-api-access-fhtlx") pod "6e111df2-b999-4ab4-bb58-e998cb5379ef" (UID: "6e111df2-b999-4ab4-bb58-e998cb5379ef"). InnerVolumeSpecName "kube-api-access-fhtlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:30:33.904867 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:33.904851 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e111df2-b999-4ab4-bb58-e998cb5379ef" (UID: "6e111df2-b999-4ab4-bb58-e998cb5379ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:30:34.003312 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.003275 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:30:34.003312 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.003308 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhtlx\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-kube-api-access-fhtlx\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:30:34.003312 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.003318 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:30:34.003548 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.003326 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:30:34.003548 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.003335 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:30:34.003548 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.003345 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:30:34.003548 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.003352 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e111df2-b999-4ab4-bb58-e998cb5379ef-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:30:34.735372 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.735341 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" Jul 15 17:30:34.735794 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.735368 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2" event={"ID":"6e111df2-b999-4ab4-bb58-e998cb5379ef","Type":"ContainerDied","Data":"025c82707c68f9b28ee43539636b0b218af3f88f2d3e0e924e2bbc868e728fbb"} Jul 15 17:30:34.735794 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.735449 2566 scope.go:117] "RemoveContainer" containerID="122e5194414204d3607072547454a7050ce8a22997bd88337cd9e015437e5932" Jul 15 17:30:34.756540 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.756510 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2"] Jul 15 17:30:34.763855 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:34.763828 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6feb2776-openshift-default-7679d4b479-sdww2"] Jul 15 17:30:36.189035 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:30:36.188998 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" path="/var/lib/kubelet/pods/6e111df2-b999-4ab4-bb58-e998cb5379ef/volumes" Jul 15 17:31:51.795032 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:31:51.794995 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:32:01.089920 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:01.089885 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:32:04.167091 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167048 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx"] Jul 15 17:32:04.167667 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167649 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" Jul 15 17:32:04.167667 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167668 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167680 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167686 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167696 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167702 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167710 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167715 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167770 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167780 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e111df2-b999-4ab4-bb58-e998cb5379ef" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167787 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" Jul 15 17:32:04.167818 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.167793 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="91b50c53-0ced-4625-b3e8-4911fe2702fe" containerName="istio-proxy" Jul 15 17:32:04.170568 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.170550 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.174120 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.174100 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4fc374e6-openshift-default-dockercfg-zm78t\"" Jul 15 17:32:04.190190 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.190162 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx"] Jul 15 17:32:04.196128 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.196097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-socket\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.196254 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.196151 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-token\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.196254 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.196176 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istiod-ca-cert\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.196254 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.196201 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6zfr\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-kube-api-access-f6zfr\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.196254 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.196230 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-podinfo\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.196428 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.196286 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-envoy\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.196428 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.196355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-certs\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.196428 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.196399 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-credential-socket\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.196557 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.196425 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-data\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297018 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.296981 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-envoy\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297235 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297042 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-certs\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297235 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-credential-socket\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297235 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297109 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-data\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297235 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297135 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-socket\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297235 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297225 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-token\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297488 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297265 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istiod-ca-cert\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297488 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297286 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6zfr\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-kube-api-access-f6zfr\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297488 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297314 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-podinfo\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297692 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297504 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-certs\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297692 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297548 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-data\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297692 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297683 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-credential-socket\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.297849 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.297807 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-socket\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.298421 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.298399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istiod-ca-cert\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.299788 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.299763 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-envoy\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.300005 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.299985 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-podinfo\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.313330 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.313300 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-token\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.314746 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.314717 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6zfr\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-kube-api-access-f6zfr\") pod \"e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.480762 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.480682 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:04.628887 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.628854 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx"] Jul 15 17:32:04.632303 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:32:04.632276 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d517e11_a74c_40c5_af24_13f3b5ca2d6b.slice/crio-9f3f5ce7c11e647d3015d3ab147ca86d35f1d5a65b5f9f8f1643af2cd5a9b7e9 WatchSource:0}: Error finding container 9f3f5ce7c11e647d3015d3ab147ca86d35f1d5a65b5f9f8f1643af2cd5a9b7e9: Status 404 returned error can't find the container with id 9f3f5ce7c11e647d3015d3ab147ca86d35f1d5a65b5f9f8f1643af2cd5a9b7e9 Jul 15 17:32:04.634493 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.634461 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:32:04.634550 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.634537 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:32:04.634590 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:04.634568 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:32:05.167209 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:05.167163 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" event={"ID":"3d517e11-a74c-40c5-af24-13f3b5ca2d6b","Type":"ContainerStarted","Data":"8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998"} Jul 15 17:32:05.167707 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:05.167218 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" event={"ID":"3d517e11-a74c-40c5-af24-13f3b5ca2d6b","Type":"ContainerStarted","Data":"9f3f5ce7c11e647d3015d3ab147ca86d35f1d5a65b5f9f8f1643af2cd5a9b7e9"} Jul 15 17:32:05.188186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:05.188125 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" podStartSLOduration=1.188105727 podStartE2EDuration="1.188105727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:32:05.187401072 +0000 UTC m=+2019.501105964" watchObservedRunningTime="2026-07-15 17:32:05.188105727 +0000 UTC m=+2019.501810620" Jul 15 17:32:05.481773 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:05.481684 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:05.486840 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:05.486813 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:06.172512 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:06.172470 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:06.173389 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:06.173370 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:11.784412 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:11.784365 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:32:20.240088 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:20.240053 2566 generic.go:356] "Generic (PLEG): container finished" podID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerID="8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998" exitCode=0 Jul 15 17:32:20.240521 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:20.240115 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" event={"ID":"3d517e11-a74c-40c5-af24-13f3b5ca2d6b","Type":"ContainerDied","Data":"8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998"} Jul 15 17:32:20.240521 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:20.240457 2566 scope.go:117] "RemoveContainer" containerID="8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998" Jul 15 17:32:20.241455 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:20.241423 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:32:20.241530 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:20.241511 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:32:20.241582 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:20.241566 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:32:21.173134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:21.173097 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:21.246978 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:21.246947 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" event={"ID":"3d517e11-a74c-40c5-af24-13f3b5ca2d6b","Type":"ContainerStarted","Data":"9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96"} Jul 15 17:32:21.481486 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:21.481392 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:21.482451 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:21.482424 2566 patch_prober.go:28] interesting pod/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 15 17:32:21.482570 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:21.482488 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 15 17:32:22.481385 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:22.481341 2566 patch_prober.go:28] interesting pod/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 15 17:32:22.481918 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:22.481406 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 15 17:32:23.481118 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:23.481079 2566 patch_prober.go:28] interesting pod/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 15 17:32:23.481307 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:23.481144 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 15 17:32:24.482201 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:24.481743 2566 patch_prober.go:28] interesting pod/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 15 17:32:24.482201 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:24.481802 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 15 17:32:25.485302 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:25.485265 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:25.485759 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:25.485553 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:25.486364 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:25.486348 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:32.784304 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:32.784261 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:32:44.755979 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:44.755943 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx"] Jul 15 17:32:44.756443 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:44.756196 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" containerID="cri-o://9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96" gracePeriod=30 Jul 15 17:32:50.015890 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.015864 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:50.115299 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115243 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-podinfo\") pod \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " Jul 15 17:32:50.115500 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115317 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-socket\") pod \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " Jul 15 17:32:50.115500 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115334 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-certs\") pod \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " Jul 15 17:32:50.115669 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115496 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6zfr\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-kube-api-access-f6zfr\") pod \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " Jul 15 17:32:50.115669 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115588 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-credential-socket\") pod \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " Jul 15 17:32:50.115669 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115596 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d517e11-a74c-40c5-af24-13f3b5ca2d6b" (UID: "3d517e11-a74c-40c5-af24-13f3b5ca2d6b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:32:50.115669 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115643 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-data\") pod \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " Jul 15 17:32:50.115882 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115679 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d517e11-a74c-40c5-af24-13f3b5ca2d6b" (UID: "3d517e11-a74c-40c5-af24-13f3b5ca2d6b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:32:50.115882 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115700 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istiod-ca-cert\") pod \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " Jul 15 17:32:50.115882 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115723 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-envoy\") pod \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " Jul 15 17:32:50.115882 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115810 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-token\") pod \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\" (UID: \"3d517e11-a74c-40c5-af24-13f3b5ca2d6b\") " Jul 15 17:32:50.116090 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.115868 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d517e11-a74c-40c5-af24-13f3b5ca2d6b" (UID: "3d517e11-a74c-40c5-af24-13f3b5ca2d6b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:32:50.116090 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.116070 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d517e11-a74c-40c5-af24-13f3b5ca2d6b" (UID: "3d517e11-a74c-40c5-af24-13f3b5ca2d6b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:32:50.116252 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.116234 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:32:50.116311 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.116257 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:32:50.116311 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.116272 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:32:50.116311 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.116287 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:32:50.116745 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.116722 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d517e11-a74c-40c5-af24-13f3b5ca2d6b" (UID: "3d517e11-a74c-40c5-af24-13f3b5ca2d6b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:32:50.117923 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.117896 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d517e11-a74c-40c5-af24-13f3b5ca2d6b" (UID: "3d517e11-a74c-40c5-af24-13f3b5ca2d6b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:32:50.118019 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.117994 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-kube-api-access-f6zfr" (OuterVolumeSpecName: "kube-api-access-f6zfr") pod "3d517e11-a74c-40c5-af24-13f3b5ca2d6b" (UID: "3d517e11-a74c-40c5-af24-13f3b5ca2d6b"). InnerVolumeSpecName "kube-api-access-f6zfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:32:50.118170 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.118149 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d517e11-a74c-40c5-af24-13f3b5ca2d6b" (UID: "3d517e11-a74c-40c5-af24-13f3b5ca2d6b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:32:50.118272 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.118241 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d517e11-a74c-40c5-af24-13f3b5ca2d6b" (UID: "3d517e11-a74c-40c5-af24-13f3b5ca2d6b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:32:50.217768 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.217735 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:32:50.217768 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.217764 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:32:50.217953 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.217777 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6zfr\" (UniqueName: \"kubernetes.io/projected/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-kube-api-access-f6zfr\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:32:50.217953 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.217788 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:32:50.217953 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.217796 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d517e11-a74c-40c5-af24-13f3b5ca2d6b-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:32:50.388207 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.388111 2566 generic.go:356] "Generic (PLEG): container finished" podID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerID="9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96" exitCode=0 Jul 15 17:32:50.388207 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.388172 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" Jul 15 17:32:50.388207 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.388193 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" event={"ID":"3d517e11-a74c-40c5-af24-13f3b5ca2d6b","Type":"ContainerDied","Data":"9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96"} Jul 15 17:32:50.388435 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.388262 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx" event={"ID":"3d517e11-a74c-40c5-af24-13f3b5ca2d6b","Type":"ContainerDied","Data":"9f3f5ce7c11e647d3015d3ab147ca86d35f1d5a65b5f9f8f1643af2cd5a9b7e9"} Jul 15 17:32:50.388435 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.388286 2566 scope.go:117] "RemoveContainer" containerID="9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96" Jul 15 17:32:50.399236 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.399191 2566 scope.go:117] "RemoveContainer" containerID="8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998" Jul 15 17:32:50.407184 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.407157 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx"] Jul 15 17:32:50.408087 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.408066 2566 scope.go:117] "RemoveContainer" containerID="9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96" Jul 15 17:32:50.408398 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:32:50.408366 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96\": container with ID starting with 9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96 not found: ID does not exist" containerID="9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96" Jul 15 17:32:50.408474 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.408408 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96"} err="failed to get container status \"9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96\": rpc error: code = NotFound desc = could not find container \"9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96\": container with ID starting with 9db7685c2a08c039d86815ea0b9a36579d3357100d98d024ef01ccfbc9884d96 not found: ID does not exist" Jul 15 17:32:50.408474 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.408426 2566 scope.go:117] "RemoveContainer" containerID="8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998" Jul 15 17:32:50.408703 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:32:50.408683 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998\": container with ID starting with 8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998 not found: ID does not exist" containerID="8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998" Jul 15 17:32:50.408759 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.408709 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998"} err="failed to get container status \"8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998\": rpc error: code = NotFound desc = could not find container \"8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998\": container with ID starting with 8b8ce4c271ed9e44de3261c86ed5e30d1a6b618663629339d746b2c84f9d5998 not found: ID does not exist" Jul 15 17:32:50.411956 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:50.411931 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4fc374e6-openshift-default-58c48cbfdc-8r4xx"] Jul 15 17:32:52.188532 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:32:52.188502 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" path="/var/lib/kubelet/pods/3d517e11-a74c-40c5-af24-13f3b5ca2d6b/volumes" Jul 15 17:33:04.478843 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.478805 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd"] Jul 15 17:33:04.479355 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.479193 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" Jul 15 17:33:04.479355 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.479214 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" Jul 15 17:33:04.479355 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.479255 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" Jul 15 17:33:04.479355 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.479262 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" Jul 15 17:33:04.479355 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.479338 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" Jul 15 17:33:04.479355 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.479348 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d517e11-a74c-40c5-af24-13f3b5ca2d6b" containerName="istio-proxy" Jul 15 17:33:04.481866 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.481831 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.484649 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.484624 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6a6deab7-openshift-default-dockercfg-4rsts\"" Jul 15 17:33:04.492920 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.492894 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd"] Jul 15 17:33:04.551824 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.551780 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-envoy\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.551994 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.551919 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl2c8\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-kube-api-access-bl2c8\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.551994 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.551971 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-credential-socket\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.552069 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.552003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-podinfo\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.552069 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.552038 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52046b0-9894-4cc5-a1be-052fb205cd22-istiod-ca-cert\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.552148 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.552089 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-token\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.552192 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.552146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-socket\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.552242 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.552228 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-certs\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.552283 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.552250 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-data\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.653479 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653447 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52046b0-9894-4cc5-a1be-052fb205cd22-istiod-ca-cert\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.653689 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653498 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-token\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.653689 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653524 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-socket\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.653689 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-certs\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.653689 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653591 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-data\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.653689 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653663 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-envoy\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.653957 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl2c8\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-kube-api-access-bl2c8\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.653957 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653740 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-credential-socket\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.653957 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653761 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-podinfo\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.654109 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.653971 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-socket\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.654109 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.654022 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-data\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.654109 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.654092 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-certs\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.654244 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.654224 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-credential-socket\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.654286 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.654250 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52046b0-9894-4cc5-a1be-052fb205cd22-istiod-ca-cert\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.656140 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.656116 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-envoy\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.656688 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.656671 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-podinfo\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.665499 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.665476 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-token\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.665624 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.665534 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl2c8\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-kube-api-access-bl2c8\") pod \"e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.794552 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.794466 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:04.937231 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.937208 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd"] Jul 15 17:33:04.939785 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:33:04.939753 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf52046b0_9894_4cc5_a1be_052fb205cd22.slice/crio-1d02fe9867cfe81379fd1081cf0bc88cd5f72c2a3a878e97fea5e6677a8b938d WatchSource:0}: Error finding container 1d02fe9867cfe81379fd1081cf0bc88cd5f72c2a3a878e97fea5e6677a8b938d: Status 404 returned error can't find the container with id 1d02fe9867cfe81379fd1081cf0bc88cd5f72c2a3a878e97fea5e6677a8b938d Jul 15 17:33:04.941883 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.941851 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:33:04.941990 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.941919 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:33:04.941990 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:04.941947 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:33:05.469142 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:05.469108 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" event={"ID":"f52046b0-9894-4cc5-a1be-052fb205cd22","Type":"ContainerStarted","Data":"6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3"} Jul 15 17:33:05.469318 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:05.469159 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" event={"ID":"f52046b0-9894-4cc5-a1be-052fb205cd22","Type":"ContainerStarted","Data":"1d02fe9867cfe81379fd1081cf0bc88cd5f72c2a3a878e97fea5e6677a8b938d"} Jul 15 17:33:05.492115 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:05.492062 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" podStartSLOduration=1.492045951 podStartE2EDuration="1.492045951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:33:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:33:05.490297673 +0000 UTC m=+2079.804002566" watchObservedRunningTime="2026-07-15 17:33:05.492045951 +0000 UTC m=+2079.805750843" Jul 15 17:33:05.795131 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:05.795038 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:05.800617 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:05.800574 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:06.474135 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:06.474099 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:06.475097 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:06.475079 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:14.091374 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:14.091341 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:33:22.292562 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:22.292325 2566 patch_prober.go:28] interesting pod/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:57344->10.132.0.45:15021: read: connection reset by peer" start-of-body= Jul 15 17:33:22.292562 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:22.292433 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": read tcp 10.132.0.2:57344->10.132.0.45:15021: read: connection reset by peer" Jul 15 17:33:22.559371 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:22.559274 2566 generic.go:356] "Generic (PLEG): container finished" podID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerID="6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3" exitCode=0 Jul 15 17:33:22.559371 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:22.559346 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" event={"ID":"f52046b0-9894-4cc5-a1be-052fb205cd22","Type":"ContainerDied","Data":"6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3"} Jul 15 17:33:22.559776 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:22.559759 2566 scope.go:117] "RemoveContainer" containerID="6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3" Jul 15 17:33:22.560863 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:22.560831 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:33:22.560927 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:22.560900 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:33:22.560961 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:22.560932 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:33:23.570872 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:23.569895 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" event={"ID":"f52046b0-9894-4cc5-a1be-052fb205cd22","Type":"ContainerStarted","Data":"5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2"} Jul 15 17:33:23.795717 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:23.795680 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:23.795934 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:23.795913 2566 patch_prober.go:28] interesting pod/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 17:33:23.795985 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:23.795964 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 17:33:24.795452 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:24.795410 2566 patch_prober.go:28] interesting pod/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 17:33:24.795904 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:24.795484 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 17:33:25.795813 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:25.795768 2566 patch_prober.go:28] interesting pod/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 17:33:25.796222 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:25.795841 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 17:33:25.899629 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:25.899569 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:33:26.293516 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:26.293485 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:33:26.299387 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:26.299359 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:33:26.300310 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:26.300289 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:33:26.305886 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:26.305867 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:33:26.795804 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:26.795763 2566 patch_prober.go:28] interesting pod/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 17:33:26.796013 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:26.795827 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 17:33:27.798665 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:27.798635 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:27.799076 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:27.799016 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:27.799695 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:27.799679 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:33:44.087009 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:33:44.086965 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:34:04.484934 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:34:04.484900 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:34:57.582787 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:34:57.582746 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd"] Jul 15 17:34:57.583313 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:34:57.583058 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" containerID="cri-o://5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2" gracePeriod=30 Jul 15 17:34:57.801233 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:34:57.801197 2566 patch_prober.go:28] interesting pod/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 17:34:57.801402 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:34:57.801279 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 17:35:01.597106 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:01.597070 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:35:02.852377 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.852309 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:35:02.921161 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921121 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-credential-socket\") pod \"f52046b0-9894-4cc5-a1be-052fb205cd22\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " Jul 15 17:35:02.921161 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921164 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-data\") pod \"f52046b0-9894-4cc5-a1be-052fb205cd22\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " Jul 15 17:35:02.921401 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921220 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-envoy\") pod \"f52046b0-9894-4cc5-a1be-052fb205cd22\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " Jul 15 17:35:02.921401 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921278 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-certs\") pod \"f52046b0-9894-4cc5-a1be-052fb205cd22\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " Jul 15 17:35:02.921401 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921327 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52046b0-9894-4cc5-a1be-052fb205cd22-istiod-ca-cert\") pod \"f52046b0-9894-4cc5-a1be-052fb205cd22\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " Jul 15 17:35:02.921401 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921368 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-podinfo\") pod \"f52046b0-9894-4cc5-a1be-052fb205cd22\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " Jul 15 17:35:02.921401 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921393 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-socket\") pod \"f52046b0-9894-4cc5-a1be-052fb205cd22\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " Jul 15 17:35:02.921702 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921425 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-token\") pod \"f52046b0-9894-4cc5-a1be-052fb205cd22\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " Jul 15 17:35:02.921702 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921470 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f52046b0-9894-4cc5-a1be-052fb205cd22" (UID: "f52046b0-9894-4cc5-a1be-052fb205cd22"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:35:02.921702 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921666 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl2c8\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-kube-api-access-bl2c8\") pod \"f52046b0-9894-4cc5-a1be-052fb205cd22\" (UID: \"f52046b0-9894-4cc5-a1be-052fb205cd22\") " Jul 15 17:35:02.921702 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921671 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f52046b0-9894-4cc5-a1be-052fb205cd22" (UID: "f52046b0-9894-4cc5-a1be-052fb205cd22"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:35:02.921921 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.921698 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f52046b0-9894-4cc5-a1be-052fb205cd22" (UID: "f52046b0-9894-4cc5-a1be-052fb205cd22"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:35:02.922084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.922016 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:35:02.922084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.922043 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:35:02.922084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.922049 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f52046b0-9894-4cc5-a1be-052fb205cd22-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f52046b0-9894-4cc5-a1be-052fb205cd22" (UID: "f52046b0-9894-4cc5-a1be-052fb205cd22"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:35:02.922084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.922059 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:35:02.922336 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.922192 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-data" (OuterVolumeSpecName: "istio-data") pod "f52046b0-9894-4cc5-a1be-052fb205cd22" (UID: "f52046b0-9894-4cc5-a1be-052fb205cd22"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:35:02.923767 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.923731 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f52046b0-9894-4cc5-a1be-052fb205cd22" (UID: "f52046b0-9894-4cc5-a1be-052fb205cd22"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:35:02.923947 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.923929 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-kube-api-access-bl2c8" (OuterVolumeSpecName: "kube-api-access-bl2c8") pod "f52046b0-9894-4cc5-a1be-052fb205cd22" (UID: "f52046b0-9894-4cc5-a1be-052fb205cd22"). InnerVolumeSpecName "kube-api-access-bl2c8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:35:02.924206 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.924182 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f52046b0-9894-4cc5-a1be-052fb205cd22" (UID: "f52046b0-9894-4cc5-a1be-052fb205cd22"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:35:02.924254 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:02.924210 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-token" (OuterVolumeSpecName: "istio-token") pod "f52046b0-9894-4cc5-a1be-052fb205cd22" (UID: "f52046b0-9894-4cc5-a1be-052fb205cd22"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:35:03.023268 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.023227 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:35:03.023268 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.023261 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f52046b0-9894-4cc5-a1be-052fb205cd22-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:35:03.023268 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.023270 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:35:03.023268 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.023278 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:35:03.023545 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.023287 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl2c8\" (UniqueName: \"kubernetes.io/projected/f52046b0-9894-4cc5-a1be-052fb205cd22-kube-api-access-bl2c8\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:35:03.023545 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.023296 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f52046b0-9894-4cc5-a1be-052fb205cd22-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:35:03.067931 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.067896 2566 generic.go:356] "Generic (PLEG): container finished" podID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerID="5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2" exitCode=0 Jul 15 17:35:03.068112 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.067949 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" Jul 15 17:35:03.068112 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.067982 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" event={"ID":"f52046b0-9894-4cc5-a1be-052fb205cd22","Type":"ContainerDied","Data":"5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2"} Jul 15 17:35:03.068112 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.068073 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd" event={"ID":"f52046b0-9894-4cc5-a1be-052fb205cd22","Type":"ContainerDied","Data":"1d02fe9867cfe81379fd1081cf0bc88cd5f72c2a3a878e97fea5e6677a8b938d"} Jul 15 17:35:03.068112 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.068102 2566 scope.go:117] "RemoveContainer" containerID="5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2" Jul 15 17:35:03.077591 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.077574 2566 scope.go:117] "RemoveContainer" containerID="6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3" Jul 15 17:35:03.086019 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.085997 2566 scope.go:117] "RemoveContainer" containerID="5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2" Jul 15 17:35:03.086270 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:35:03.086252 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2\": container with ID starting with 5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2 not found: ID does not exist" containerID="5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2" Jul 15 17:35:03.086314 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.086280 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2"} err="failed to get container status \"5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2\": rpc error: code = NotFound desc = could not find container \"5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2\": container with ID starting with 5d8dd96b0576354b653a8d42a04b6491db77eae128a23b72f669eeb3c256baa2 not found: ID does not exist" Jul 15 17:35:03.086314 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.086295 2566 scope.go:117] "RemoveContainer" containerID="6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3" Jul 15 17:35:03.086493 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:35:03.086479 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3\": container with ID starting with 6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3 not found: ID does not exist" containerID="6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3" Jul 15 17:35:03.086530 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.086498 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3"} err="failed to get container status \"6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3\": rpc error: code = NotFound desc = could not find container \"6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3\": container with ID starting with 6f00d151bfb616667dd524c292318e6cf5f114fc17e64fdabfded507327549c3 not found: ID does not exist" Jul 15 17:35:03.094005 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.093973 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd"] Jul 15 17:35:03.098625 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:03.098586 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6a6deab7-openshift-default-84679cb57f-csjjd"] Jul 15 17:35:04.188405 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:04.188369 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" path="/var/lib/kubelet/pods/f52046b0-9894-4cc5-a1be-052fb205cd22/volumes" Jul 15 17:35:15.991017 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:15.990981 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:35:52.434204 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.434102 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf"] Jul 15 17:35:52.434784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.434636 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" Jul 15 17:35:52.434784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.434652 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" Jul 15 17:35:52.434784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.434661 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" Jul 15 17:35:52.434784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.434667 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" Jul 15 17:35:52.434784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.434751 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" Jul 15 17:35:52.434784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.434759 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f52046b0-9894-4cc5-a1be-052fb205cd22" containerName="istio-proxy" Jul 15 17:35:52.436940 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.436917 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.439757 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.439733 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-a6dd94-openshift-default-dockercfg-6blpw\"" Jul 15 17:35:52.452066 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.452043 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf"] Jul 15 17:35:52.573694 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.573648 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-certs\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.573875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.573707 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-podinfo\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.573875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.573738 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqpsl\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-kube-api-access-nqpsl\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.573875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.573832 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146920f4-5449-4c75-a190-bbb52a8ba9dc-istiod-ca-cert\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.573982 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.573891 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-data\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.573982 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.573917 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-token\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.573982 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.573937 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-credential-socket\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.573982 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.573952 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-envoy\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.574139 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.574048 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-socket\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675023 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.674987 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-socket\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675212 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675041 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-certs\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675212 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-podinfo\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675212 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675198 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqpsl\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-kube-api-access-nqpsl\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675329 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675254 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146920f4-5449-4c75-a190-bbb52a8ba9dc-istiod-ca-cert\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675329 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675313 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-data\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675425 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675347 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-token\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675425 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675382 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-credential-socket\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675425 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675404 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-envoy\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675685 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675659 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-certs\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.675874 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675852 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-socket\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.676005 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.675976 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-credential-socket\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.676069 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.676002 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-data\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.676376 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.676352 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146920f4-5449-4c75-a190-bbb52a8ba9dc-istiod-ca-cert\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.677876 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.677852 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-envoy\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.678141 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.678126 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-podinfo\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.683100 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.683079 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-token\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.683232 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.683215 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqpsl\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-kube-api-access-nqpsl\") pod \"e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.749499 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.749410 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:52.896473 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.896447 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf"] Jul 15 17:35:52.898162 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:35:52.898128 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod146920f4_5449_4c75_a190_bbb52a8ba9dc.slice/crio-20872a66d57d23a932b4521fee70398c83a9b8f6c34c0e33c1a124cb6f44dfe8 WatchSource:0}: Error finding container 20872a66d57d23a932b4521fee70398c83a9b8f6c34c0e33c1a124cb6f44dfe8: Status 404 returned error can't find the container with id 20872a66d57d23a932b4521fee70398c83a9b8f6c34c0e33c1a124cb6f44dfe8 Jul 15 17:35:52.900388 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.900354 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:35:52.900468 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.900418 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:35:52.900468 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:52.900449 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:35:53.310324 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:53.310289 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" event={"ID":"146920f4-5449-4c75-a190-bbb52a8ba9dc","Type":"ContainerStarted","Data":"db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3"} Jul 15 17:35:53.310505 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:53.310339 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" event={"ID":"146920f4-5449-4c75-a190-bbb52a8ba9dc","Type":"ContainerStarted","Data":"20872a66d57d23a932b4521fee70398c83a9b8f6c34c0e33c1a124cb6f44dfe8"} Jul 15 17:35:53.331403 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:53.331352 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" podStartSLOduration=1.3313373020000001 podStartE2EDuration="1.331337302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:35:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:35:53.330069066 +0000 UTC m=+2247.643773973" watchObservedRunningTime="2026-07-15 17:35:53.331337302 +0000 UTC m=+2247.645042193" Jul 15 17:35:53.750505 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:53.750468 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:53.755333 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:53.755304 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:54.315743 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:54.315718 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:35:54.316645 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:35:54.316595 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:36:00.361875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.361836 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv"] Jul 15 17:36:00.364489 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.364465 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.367105 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.367083 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-47c774-openshift-default-dockercfg-9qqx7\"" Jul 15 17:36:00.375864 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.375814 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv"] Jul 15 17:36:00.441933 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.441896 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-certs\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.441933 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.441941 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-credential-socket\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.442205 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.441961 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-envoy\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.442205 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.442051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-istio-token\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.442205 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.442103 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1e7938-8958-4211-abd7-95488f073e9b-istio-podinfo\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.442205 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.442144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-data\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.442205 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.442171 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1e7938-8958-4211-abd7-95488f073e9b-istiod-ca-cert\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.442441 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.442267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-socket\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.442441 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.442299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7vsf\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-kube-api-access-w7vsf\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.543629 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.543574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1e7938-8958-4211-abd7-95488f073e9b-istio-podinfo\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.543802 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.543669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-data\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.543802 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.543700 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1e7938-8958-4211-abd7-95488f073e9b-istiod-ca-cert\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.543802 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.543777 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-socket\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.543802 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.543798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7vsf\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-kube-api-access-w7vsf\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.544093 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.543863 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-certs\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.544194 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.544174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-credential-socket\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.544347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.544330 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-envoy\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.544497 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.544481 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-istio-token\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.544686 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.544528 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-certs\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.544815 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.544530 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-credential-socket\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.544946 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.544185 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-data\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.545052 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.544709 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1e7938-8958-4211-abd7-95488f073e9b-istiod-ca-cert\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.545325 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.544813 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-socket\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.546851 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.546826 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-envoy\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.546934 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.546867 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1e7938-8958-4211-abd7-95488f073e9b-istio-podinfo\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.554903 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.554530 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7vsf\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-kube-api-access-w7vsf\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.555134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.554929 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-istio-token\") pod \"e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.678519 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.678426 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:00.822634 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.822584 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv"] Jul 15 17:36:00.825434 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:36:00.825404 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e1e7938_8958_4211_abd7_95488f073e9b.slice/crio-057c06f96c1dcf2cef578e1c2a9ba4d1432cf47dccf470d740b693ff611fae6b WatchSource:0}: Error finding container 057c06f96c1dcf2cef578e1c2a9ba4d1432cf47dccf470d740b693ff611fae6b: Status 404 returned error can't find the container with id 057c06f96c1dcf2cef578e1c2a9ba4d1432cf47dccf470d740b693ff611fae6b Jul 15 17:36:00.827707 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.827671 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:36:00.827812 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.827756 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:36:00.827812 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:00.827802 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:36:01.351682 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:01.351644 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" event={"ID":"6e1e7938-8958-4211-abd7-95488f073e9b","Type":"ContainerStarted","Data":"cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840"} Jul 15 17:36:01.351877 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:01.351707 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" event={"ID":"6e1e7938-8958-4211-abd7-95488f073e9b","Type":"ContainerStarted","Data":"057c06f96c1dcf2cef578e1c2a9ba4d1432cf47dccf470d740b693ff611fae6b"} Jul 15 17:36:01.376945 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:01.376893 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" podStartSLOduration=1.376874202 podStartE2EDuration="1.376874202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:36:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:36:01.374261306 +0000 UTC m=+2255.687966199" watchObservedRunningTime="2026-07-15 17:36:01.376874202 +0000 UTC m=+2255.690579094" Jul 15 17:36:01.679077 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:01.678984 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:01.684190 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:01.684166 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:02.357237 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:02.357207 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:02.358171 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:02.358148 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:03.907377 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:03.907341 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:36:10.357825 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:10.357771 2566 patch_prober.go:28] interesting pod/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 17:36:10.358322 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:10.357845 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 17:36:12.413786 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:12.413749 2566 generic.go:356] "Generic (PLEG): container finished" podID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerID="db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3" exitCode=0 Jul 15 17:36:12.414345 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:12.413824 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" event={"ID":"146920f4-5449-4c75-a190-bbb52a8ba9dc","Type":"ContainerDied","Data":"db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3"} Jul 15 17:36:12.414424 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:12.414343 2566 scope.go:117] "RemoveContainer" containerID="db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3" Jul 15 17:36:12.415400 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:12.415374 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:36:12.415465 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:12.415439 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:36:12.415501 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:12.415470 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:36:13.421007 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:13.420968 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" event={"ID":"146920f4-5449-4c75-a190-bbb52a8ba9dc","Type":"ContainerStarted","Data":"ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e"} Jul 15 17:36:13.749861 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:13.749774 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:36:13.750017 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:13.749988 2566 patch_prober.go:28] interesting pod/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 15 17:36:13.750063 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:13.750031 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 15 17:36:14.487217 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:14.487166 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:36:14.750923 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:14.750837 2566 patch_prober.go:28] interesting pod/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 15 17:36:14.750923 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:14.750889 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 15 17:36:15.750517 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:15.750471 2566 patch_prober.go:28] interesting pod/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 15 17:36:15.750926 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:15.750546 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 15 17:36:16.750085 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:16.750041 2566 patch_prober.go:28] interesting pod/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 15 17:36:16.750296 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:16.750111 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 15 17:36:17.753727 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:17.753696 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:36:17.754161 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:17.753971 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:36:17.754774 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:17.754757 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:36:18.399805 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:18.399764 2566 patch_prober.go:28] interesting pod/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 17:36:18.399984 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:18.399826 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 17:36:19.452681 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:19.452642 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e1e7938-8958-4211-abd7-95488f073e9b" containerID="cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840" exitCode=0 Jul 15 17:36:19.453215 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:19.452721 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" event={"ID":"6e1e7938-8958-4211-abd7-95488f073e9b","Type":"ContainerDied","Data":"cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840"} Jul 15 17:36:19.453292 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:19.453270 2566 scope.go:117] "RemoveContainer" containerID="cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840" Jul 15 17:36:19.454261 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:19.454232 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:36:19.454355 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:19.454297 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:36:19.454355 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:19.454330 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:36:20.461971 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:20.461911 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" event={"ID":"6e1e7938-8958-4211-abd7-95488f073e9b","Type":"ContainerStarted","Data":"cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde"} Jul 15 17:36:20.679742 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:20.679700 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:20.679924 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:20.679893 2566 patch_prober.go:28] interesting pod/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 17:36:20.679987 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:20.679955 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 17:36:21.679530 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:21.679487 2566 patch_prober.go:28] interesting pod/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 17:36:21.680084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:21.679559 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 17:36:22.678934 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:22.678894 2566 patch_prober.go:28] interesting pod/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 17:36:22.679277 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:22.679240 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 17:36:22.794298 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:22.794256 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:36:23.679566 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:23.679092 2566 patch_prober.go:28] interesting pod/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 17:36:23.679566 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:23.679161 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 17:36:24.682261 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:24.682232 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:24.682732 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:24.682516 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:24.683372 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:24.683356 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:36:33.995058 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:33.995017 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:36:48.296822 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:36:48.296782 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:37:21.195592 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:21.195558 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:37:34.394665 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:34.394627 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:37:43.545771 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:43.545577 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf"] Jul 15 17:37:43.546272 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:43.545859 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" containerID="cri-o://ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e" gracePeriod=30 Jul 15 17:37:47.756684 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:47.756636 2566 patch_prober.go:28] interesting pod/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 17:37:47.757060 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:47.756702 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 17:37:48.808047 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.807984 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:37:48.871859 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.871825 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqpsl\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-kube-api-access-nqpsl\") pod \"146920f4-5449-4c75-a190-bbb52a8ba9dc\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " Jul 15 17:37:48.872023 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.871903 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-socket\") pod \"146920f4-5449-4c75-a190-bbb52a8ba9dc\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " Jul 15 17:37:48.872023 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872005 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-token\") pod \"146920f4-5449-4c75-a190-bbb52a8ba9dc\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " Jul 15 17:37:48.872094 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872065 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-certs\") pod \"146920f4-5449-4c75-a190-bbb52a8ba9dc\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " Jul 15 17:37:48.872127 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872091 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-podinfo\") pod \"146920f4-5449-4c75-a190-bbb52a8ba9dc\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " Jul 15 17:37:48.872127 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872121 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146920f4-5449-4c75-a190-bbb52a8ba9dc-istiod-ca-cert\") pod \"146920f4-5449-4c75-a190-bbb52a8ba9dc\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " Jul 15 17:37:48.872225 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872162 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-data\") pod \"146920f4-5449-4c75-a190-bbb52a8ba9dc\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " Jul 15 17:37:48.872225 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872164 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "146920f4-5449-4c75-a190-bbb52a8ba9dc" (UID: "146920f4-5449-4c75-a190-bbb52a8ba9dc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:37:48.872225 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872188 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-envoy\") pod \"146920f4-5449-4c75-a190-bbb52a8ba9dc\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " Jul 15 17:37:48.872225 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872217 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-credential-socket\") pod \"146920f4-5449-4c75-a190-bbb52a8ba9dc\" (UID: \"146920f4-5449-4c75-a190-bbb52a8ba9dc\") " Jul 15 17:37:48.872431 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872395 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "146920f4-5449-4c75-a190-bbb52a8ba9dc" (UID: "146920f4-5449-4c75-a190-bbb52a8ba9dc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:37:48.872591 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872543 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:37:48.872701 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872596 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:37:48.872701 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872632 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "146920f4-5449-4c75-a190-bbb52a8ba9dc" (UID: "146920f4-5449-4c75-a190-bbb52a8ba9dc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:37:48.872976 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.872728 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/146920f4-5449-4c75-a190-bbb52a8ba9dc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "146920f4-5449-4c75-a190-bbb52a8ba9dc" (UID: "146920f4-5449-4c75-a190-bbb52a8ba9dc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:37:48.873311 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.873285 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-data" (OuterVolumeSpecName: "istio-data") pod "146920f4-5449-4c75-a190-bbb52a8ba9dc" (UID: "146920f4-5449-4c75-a190-bbb52a8ba9dc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:37:48.874333 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.874306 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "146920f4-5449-4c75-a190-bbb52a8ba9dc" (UID: "146920f4-5449-4c75-a190-bbb52a8ba9dc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:37:48.874442 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.874330 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-token" (OuterVolumeSpecName: "istio-token") pod "146920f4-5449-4c75-a190-bbb52a8ba9dc" (UID: "146920f4-5449-4c75-a190-bbb52a8ba9dc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:37:48.874499 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.874433 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "146920f4-5449-4c75-a190-bbb52a8ba9dc" (UID: "146920f4-5449-4c75-a190-bbb52a8ba9dc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:37:48.874499 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.874440 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-kube-api-access-nqpsl" (OuterVolumeSpecName: "kube-api-access-nqpsl") pod "146920f4-5449-4c75-a190-bbb52a8ba9dc" (UID: "146920f4-5449-4c75-a190-bbb52a8ba9dc"). InnerVolumeSpecName "kube-api-access-nqpsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:37:48.915730 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.915696 2566 generic.go:356] "Generic (PLEG): container finished" podID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerID="ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e" exitCode=0 Jul 15 17:37:48.915925 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.915765 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" Jul 15 17:37:48.915925 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.915775 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" event={"ID":"146920f4-5449-4c75-a190-bbb52a8ba9dc","Type":"ContainerDied","Data":"ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e"} Jul 15 17:37:48.915925 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.915830 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf" event={"ID":"146920f4-5449-4c75-a190-bbb52a8ba9dc","Type":"ContainerDied","Data":"20872a66d57d23a932b4521fee70398c83a9b8f6c34c0e33c1a124cb6f44dfe8"} Jul 15 17:37:48.915925 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.915852 2566 scope.go:117] "RemoveContainer" containerID="ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e" Jul 15 17:37:48.925387 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.925367 2566 scope.go:117] "RemoveContainer" containerID="db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3" Jul 15 17:37:48.933911 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.933890 2566 scope.go:117] "RemoveContainer" containerID="ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e" Jul 15 17:37:48.934196 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:37:48.934175 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e\": container with ID starting with ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e not found: ID does not exist" containerID="ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e" Jul 15 17:37:48.934239 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.934205 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e"} err="failed to get container status \"ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e\": rpc error: code = NotFound desc = could not find container \"ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e\": container with ID starting with ea08f3c4cc92716939bb58bee8284ff2bb147641ed80e033c19493ff0eea146e not found: ID does not exist" Jul 15 17:37:48.934239 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.934225 2566 scope.go:117] "RemoveContainer" containerID="db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3" Jul 15 17:37:48.934488 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:37:48.934470 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3\": container with ID starting with db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3 not found: ID does not exist" containerID="db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3" Jul 15 17:37:48.934529 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.934494 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3"} err="failed to get container status \"db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3\": rpc error: code = NotFound desc = could not find container \"db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3\": container with ID starting with db9f5170c360f208946f5c38d35477c3bc5f5f45826aa63b389b5b83c0a738b3 not found: ID does not exist" Jul 15 17:37:48.939164 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.939141 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf"] Jul 15 17:37:48.943087 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.943063 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-a6dd94-openshift-default-dcff86b57-7ttzf"] Jul 15 17:37:48.973987 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.973950 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:37:48.973987 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.973980 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:37:48.973987 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.973990 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/146920f4-5449-4c75-a190-bbb52a8ba9dc-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:37:48.974172 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.974001 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:37:48.974172 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.974008 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:37:48.974172 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.974015 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/146920f4-5449-4c75-a190-bbb52a8ba9dc-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:37:48.974172 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:48.974025 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqpsl\" (UniqueName: \"kubernetes.io/projected/146920f4-5449-4c75-a190-bbb52a8ba9dc-kube-api-access-nqpsl\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:37:50.187896 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:50.187865 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" path="/var/lib/kubelet/pods/146920f4-5449-4c75-a190-bbb52a8ba9dc/volumes" Jul 15 17:37:57.700384 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:37:57.700347 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:38:09.732134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:09.732100 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv"] Jul 15 17:38:09.732667 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:09.732329 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" containerID="cri-o://cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde" gracePeriod=30 Jul 15 17:38:10.618344 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.618309 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6"] Jul 15 17:38:10.618745 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.618720 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" Jul 15 17:38:10.618745 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.618738 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" Jul 15 17:38:10.618745 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.618749 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" Jul 15 17:38:10.618949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.618756 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" Jul 15 17:38:10.618949 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.618869 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" Jul 15 17:38:10.621059 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.621042 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.623840 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.623820 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4dfdb7-openshift-default-dockercfg-ntkj4\"" Jul 15 17:38:10.637839 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.637814 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6"] Jul 15 17:38:10.774838 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.774801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-envoy\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.775250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.774845 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-certs\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.775250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.774871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-credential-socket\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.775250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.774938 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-podinfo\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.775250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.775014 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-token\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.775250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.775063 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-socket\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.775250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.775108 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-data\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.775250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.775141 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlj6z\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-kube-api-access-wlj6z\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.775250 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.775184 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0936eac3-9d11-4e29-a2a5-71de6af4424a-istiod-ca-cert\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876118 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876013 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0936eac3-9d11-4e29-a2a5-71de6af4424a-istiod-ca-cert\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876118 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876072 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-envoy\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876118 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-certs\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876412 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876143 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-credential-socket\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876412 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876172 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-podinfo\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876412 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876200 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-token\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876412 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876250 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-socket\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876412 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-data\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876412 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876338 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlj6z\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-kube-api-access-wlj6z\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876745 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876537 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-certs\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876745 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876663 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-socket\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876745 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876714 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-data\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.876873 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.876846 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0936eac3-9d11-4e29-a2a5-71de6af4424a-istiod-ca-cert\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.877330 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.877247 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-credential-socket\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.878623 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.878586 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-envoy\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.878952 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.878935 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-podinfo\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.891960 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.891932 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-token\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.898131 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.898104 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlj6z\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-kube-api-access-wlj6z\") pod \"e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:10.932213 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:10.932175 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:11.088226 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:11.088203 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6"] Jul 15 17:38:11.090634 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:38:11.090586 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0936eac3_9d11_4e29_a2a5_71de6af4424a.slice/crio-9c18e4d931d02f830a36de767aa0a372b7fe60bd610bc4571c6accc34ec58c9e WatchSource:0}: Error finding container 9c18e4d931d02f830a36de767aa0a372b7fe60bd610bc4571c6accc34ec58c9e: Status 404 returned error can't find the container with id 9c18e4d931d02f830a36de767aa0a372b7fe60bd610bc4571c6accc34ec58c9e Jul 15 17:38:11.092845 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:11.092812 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:11.092913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:11.092878 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:11.092913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:11.092907 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:12.034404 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:12.034372 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" event={"ID":"0936eac3-9d11-4e29-a2a5-71de6af4424a","Type":"ContainerStarted","Data":"167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b"} Jul 15 17:38:12.034830 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:12.034426 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" event={"ID":"0936eac3-9d11-4e29-a2a5-71de6af4424a","Type":"ContainerStarted","Data":"9c18e4d931d02f830a36de767aa0a372b7fe60bd610bc4571c6accc34ec58c9e"} Jul 15 17:38:12.061006 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:12.060956 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" podStartSLOduration=2.060938625 podStartE2EDuration="2.060938625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:38:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:38:12.059464542 +0000 UTC m=+2386.373169433" watchObservedRunningTime="2026-07-15 17:38:12.060938625 +0000 UTC m=+2386.374643518" Jul 15 17:38:12.932856 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:12.932814 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:12.938088 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:12.938059 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:13.038982 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:13.038954 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:13.039993 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:13.039971 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:14.993822 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:14.993761 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:38:15.054252 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.054216 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e1e7938-8958-4211-abd7-95488f073e9b" containerID="cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde" exitCode=0 Jul 15 17:38:15.054451 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.054271 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" Jul 15 17:38:15.054451 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.054298 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" event={"ID":"6e1e7938-8958-4211-abd7-95488f073e9b","Type":"ContainerDied","Data":"cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde"} Jul 15 17:38:15.054451 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.054373 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv" event={"ID":"6e1e7938-8958-4211-abd7-95488f073e9b","Type":"ContainerDied","Data":"057c06f96c1dcf2cef578e1c2a9ba4d1432cf47dccf470d740b693ff611fae6b"} Jul 15 17:38:15.054451 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.054408 2566 scope.go:117] "RemoveContainer" containerID="cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde" Jul 15 17:38:15.065653 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.065627 2566 scope.go:117] "RemoveContainer" containerID="cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840" Jul 15 17:38:15.077097 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.077075 2566 scope.go:117] "RemoveContainer" containerID="cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde" Jul 15 17:38:15.077460 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:38:15.077439 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde\": container with ID starting with cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde not found: ID does not exist" containerID="cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde" Jul 15 17:38:15.077551 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.077470 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde"} err="failed to get container status \"cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde\": rpc error: code = NotFound desc = could not find container \"cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde\": container with ID starting with cebf6c1b3b26e6443b83f654862cdb3ace0723d9ce7e57f99515757d655d6cde not found: ID does not exist" Jul 15 17:38:15.077551 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.077490 2566 scope.go:117] "RemoveContainer" containerID="cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840" Jul 15 17:38:15.077820 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:38:15.077800 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840\": container with ID starting with cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840 not found: ID does not exist" containerID="cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840" Jul 15 17:38:15.077878 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.077826 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840"} err="failed to get container status \"cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840\": rpc error: code = NotFound desc = could not find container \"cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840\": container with ID starting with cc42a38f350a798834a0912e9083e069c605d49b1ca6783686042b72b898a840 not found: ID does not exist" Jul 15 17:38:15.117695 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.117646 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w7vsf\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-kube-api-access-w7vsf\") pod \"6e1e7938-8958-4211-abd7-95488f073e9b\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " Jul 15 17:38:15.117870 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.117706 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-certs\") pod \"6e1e7938-8958-4211-abd7-95488f073e9b\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " Jul 15 17:38:15.117870 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.117754 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1e7938-8958-4211-abd7-95488f073e9b-istio-podinfo\") pod \"6e1e7938-8958-4211-abd7-95488f073e9b\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " Jul 15 17:38:15.117870 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.117793 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1e7938-8958-4211-abd7-95488f073e9b-istiod-ca-cert\") pod \"6e1e7938-8958-4211-abd7-95488f073e9b\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " Jul 15 17:38:15.117870 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.117853 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-envoy\") pod \"6e1e7938-8958-4211-abd7-95488f073e9b\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " Jul 15 17:38:15.118067 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.117882 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-credential-socket\") pod \"6e1e7938-8958-4211-abd7-95488f073e9b\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " Jul 15 17:38:15.118067 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.117919 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-socket\") pod \"6e1e7938-8958-4211-abd7-95488f073e9b\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " Jul 15 17:38:15.118067 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.117959 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-istio-token\") pod \"6e1e7938-8958-4211-abd7-95488f073e9b\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " Jul 15 17:38:15.118067 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.117990 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-data\") pod \"6e1e7938-8958-4211-abd7-95488f073e9b\" (UID: \"6e1e7938-8958-4211-abd7-95488f073e9b\") " Jul 15 17:38:15.118067 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.118021 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e1e7938-8958-4211-abd7-95488f073e9b" (UID: "6e1e7938-8958-4211-abd7-95488f073e9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:15.118312 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.118208 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e1e7938-8958-4211-abd7-95488f073e9b" (UID: "6e1e7938-8958-4211-abd7-95488f073e9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:15.118312 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.118267 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e1e7938-8958-4211-abd7-95488f073e9b" (UID: "6e1e7938-8958-4211-abd7-95488f073e9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:15.118432 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.118335 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e1e7938-8958-4211-abd7-95488f073e9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e1e7938-8958-4211-abd7-95488f073e9b" (UID: "6e1e7938-8958-4211-abd7-95488f073e9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:38:15.118432 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.118418 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:38:15.118539 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.118437 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:38:15.118539 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.118452 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:38:15.119186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.119159 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e1e7938-8958-4211-abd7-95488f073e9b" (UID: "6e1e7938-8958-4211-abd7-95488f073e9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:15.120306 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.120275 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e1e7938-8958-4211-abd7-95488f073e9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e1e7938-8958-4211-abd7-95488f073e9b" (UID: "6e1e7938-8958-4211-abd7-95488f073e9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:38:15.120306 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.120285 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e1e7938-8958-4211-abd7-95488f073e9b" (UID: "6e1e7938-8958-4211-abd7-95488f073e9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:38:15.120306 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.120300 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-kube-api-access-w7vsf" (OuterVolumeSpecName: "kube-api-access-w7vsf") pod "6e1e7938-8958-4211-abd7-95488f073e9b" (UID: "6e1e7938-8958-4211-abd7-95488f073e9b"). InnerVolumeSpecName "kube-api-access-w7vsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:38:15.120523 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.120440 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e1e7938-8958-4211-abd7-95488f073e9b" (UID: "6e1e7938-8958-4211-abd7-95488f073e9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:38:15.219893 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.219842 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7vsf\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-kube-api-access-w7vsf\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:38:15.219893 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.219878 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e1e7938-8958-4211-abd7-95488f073e9b-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:38:15.220134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.219894 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e1e7938-8958-4211-abd7-95488f073e9b-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:38:15.220134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.219923 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:38:15.220134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.219936 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e1e7938-8958-4211-abd7-95488f073e9b-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:38:15.220134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.219947 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e1e7938-8958-4211-abd7-95488f073e9b-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:38:15.378089 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.378059 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv"] Jul 15 17:38:15.382505 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:15.382479 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-47c774-openshift-default-69c96c89cf-hktnv"] Jul 15 17:38:16.188500 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:16.188465 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" path="/var/lib/kubelet/pods/6e1e7938-8958-4211-abd7-95488f073e9b/volumes" Jul 15 17:38:18.190467 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:18.190437 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:38:19.288558 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.288525 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp"] Jul 15 17:38:19.289114 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.289091 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" Jul 15 17:38:19.289185 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.289119 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" Jul 15 17:38:19.289185 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.289152 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" Jul 15 17:38:19.289185 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.289160 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" Jul 15 17:38:19.289344 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.289274 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="146920f4-5449-4c75-a190-bbb52a8ba9dc" containerName="istio-proxy" Jul 15 17:38:19.289344 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.289285 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" Jul 15 17:38:19.289344 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.289292 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e1e7938-8958-4211-abd7-95488f073e9b" containerName="istio-proxy" Jul 15 17:38:19.292329 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.292308 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.295275 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.295242 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-feaa2b-openshift-default-dockercfg-q6knl\"" Jul 15 17:38:19.304622 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.304573 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp"] Jul 15 17:38:19.457694 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.457650 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-podinfo\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.457879 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.457716 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-credential-socket\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.457879 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.457748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-socket\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.457879 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.457786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9hvw\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-kube-api-access-z9hvw\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.457879 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.457852 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-envoy\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.458047 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.457879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86175ad4-281c-4a41-ba79-eab7c5abbecc-istiod-ca-cert\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.458047 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.457913 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-token\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.458047 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.457952 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-certs\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.458047 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.457973 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-data\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.559589 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.559498 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-podinfo\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.559589 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.559561 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-credential-socket\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.559859 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.559623 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-socket\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.559859 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.559662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9hvw\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-kube-api-access-z9hvw\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.559859 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.559735 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-envoy\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.559859 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.559758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86175ad4-281c-4a41-ba79-eab7c5abbecc-istiod-ca-cert\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.559859 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.559792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-token\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.559859 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.559837 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-certs\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.559859 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.559862 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-data\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.560343 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.560016 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-credential-socket\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.560343 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.560207 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-data\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.560502 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.560480 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-certs\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.560808 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.560789 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-socket\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.560955 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.560938 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86175ad4-281c-4a41-ba79-eab7c5abbecc-istiod-ca-cert\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.562235 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.562208 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-podinfo\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.562329 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.562275 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-envoy\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.574367 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.574339 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-token\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.584593 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.584563 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9hvw\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-kube-api-access-z9hvw\") pod \"e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.603547 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.603507 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:19.758492 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.758458 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp"] Jul 15 17:38:19.760215 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:38:19.760186 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86175ad4_281c_4a41_ba79_eab7c5abbecc.slice/crio-3cbd5f0e8dd257861265b922f7eb9e08854b4f6f8d80dffcc2fc8bde53935631 WatchSource:0}: Error finding container 3cbd5f0e8dd257861265b922f7eb9e08854b4f6f8d80dffcc2fc8bde53935631: Status 404 returned error can't find the container with id 3cbd5f0e8dd257861265b922f7eb9e08854b4f6f8d80dffcc2fc8bde53935631 Jul 15 17:38:19.762762 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.762725 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:19.762838 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.762822 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:19.762885 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:19.762866 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:20.085752 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:20.085710 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" event={"ID":"86175ad4-281c-4a41-ba79-eab7c5abbecc","Type":"ContainerStarted","Data":"a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1"} Jul 15 17:38:20.085972 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:20.085782 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" event={"ID":"86175ad4-281c-4a41-ba79-eab7c5abbecc","Type":"ContainerStarted","Data":"3cbd5f0e8dd257861265b922f7eb9e08854b4f6f8d80dffcc2fc8bde53935631"} Jul 15 17:38:20.115387 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:20.115328 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" podStartSLOduration=1.115311253 podStartE2EDuration="1.115311253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:38:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:38:20.114074518 +0000 UTC m=+2394.427779412" watchObservedRunningTime="2026-07-15 17:38:20.115311253 +0000 UTC m=+2394.429016197" Jul 15 17:38:20.604271 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:20.604230 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:20.608824 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:20.608799 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:21.091266 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:21.091222 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:21.092349 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:21.092330 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:26.342411 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:26.342379 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:38:26.354181 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:26.354149 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:38:26.366399 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:26.366356 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:38:26.378075 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:26.378027 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:38:28.040235 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:28.040193 2566 patch_prober.go:28] interesting pod/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 17:38:28.040708 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:28.040251 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 17:38:28.127832 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:28.127800 2566 generic.go:356] "Generic (PLEG): container finished" podID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerID="167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b" exitCode=0 Jul 15 17:38:28.128015 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:28.127887 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" event={"ID":"0936eac3-9d11-4e29-a2a5-71de6af4424a","Type":"ContainerDied","Data":"167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b"} Jul 15 17:38:28.128378 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:28.128360 2566 scope.go:117] "RemoveContainer" containerID="167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b" Jul 15 17:38:28.129391 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:28.129362 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:28.129464 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:28.129446 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:28.129501 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:28.129487 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:29.134856 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:29.134825 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" event={"ID":"0936eac3-9d11-4e29-a2a5-71de6af4424a","Type":"ContainerStarted","Data":"803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876"} Jul 15 17:38:29.932700 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:29.932647 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:29.932976 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:29.932957 2566 patch_prober.go:28] interesting pod/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 17:38:29.933024 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:29.933001 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 17:38:30.933495 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:30.933457 2566 patch_prober.go:28] interesting pod/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 17:38:30.934004 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:30.933536 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 17:38:31.933389 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:31.933341 2566 patch_prober.go:28] interesting pod/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 17:38:31.933569 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:31.933420 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 17:38:32.706467 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:32.706429 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:38:32.932911 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:32.932870 2566 patch_prober.go:28] interesting pod/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 17:38:32.933084 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:32.932929 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 17:38:33.932838 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:33.932790 2566 patch_prober.go:28] interesting pod/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 17:38:33.933231 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:33.932877 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 17:38:34.933314 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:34.933280 2566 patch_prober.go:28] interesting pod/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 15 17:38:34.933760 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:34.933328 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 15 17:38:35.177111 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:35.177075 2566 generic.go:356] "Generic (PLEG): container finished" podID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerID="a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1" exitCode=0 Jul 15 17:38:35.177291 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:35.177152 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" event={"ID":"86175ad4-281c-4a41-ba79-eab7c5abbecc","Type":"ContainerDied","Data":"a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1"} Jul 15 17:38:35.177637 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:35.177588 2566 scope.go:117] "RemoveContainer" containerID="a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1" Jul 15 17:38:35.178666 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:35.178633 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:35.178784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:35.178718 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:35.178784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:35.178765 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:38:35.936786 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:35.936758 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:35.937206 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:35.937187 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:35.937899 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:35.937881 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:38:36.091442 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:36.091403 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:36.194320 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:36.194221 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" event={"ID":"86175ad4-281c-4a41-ba79-eab7c5abbecc","Type":"ContainerStarted","Data":"573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649"} Jul 15 17:38:36.604510 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:36.604469 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:36.605792 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:36.605766 2566 patch_prober.go:28] interesting pod/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 17:38:36.605914 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:36.605819 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 17:38:37.604186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:37.604148 2566 patch_prober.go:28] interesting pod/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 17:38:37.604576 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:37.604202 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 17:38:38.604347 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:38.604310 2566 patch_prober.go:28] interesting pod/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 17:38:38.604775 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:38.604372 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 17:38:39.604597 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:39.604556 2566 patch_prober.go:28] interesting pod/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 17:38:39.605062 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:39.604651 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 17:38:40.608735 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:40.608703 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:40.609130 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:40.608973 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:40.609595 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:40.609580 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:38:49.497982 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:38:49.497884 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:39:00.549427 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:00.549387 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6"] Jul 15 17:39:00.550092 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:00.549672 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" containerID="cri-o://803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876" gracePeriod=30 Jul 15 17:39:05.812101 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.812034 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:39:05.920883 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.920850 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-podinfo\") pod \"0936eac3-9d11-4e29-a2a5-71de6af4424a\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " Jul 15 17:39:05.921080 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.920951 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-token\") pod \"0936eac3-9d11-4e29-a2a5-71de6af4424a\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " Jul 15 17:39:05.921080 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.920973 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-socket\") pod \"0936eac3-9d11-4e29-a2a5-71de6af4424a\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " Jul 15 17:39:05.921080 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.920998 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlj6z\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-kube-api-access-wlj6z\") pod \"0936eac3-9d11-4e29-a2a5-71de6af4424a\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " Jul 15 17:39:05.921080 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921017 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-credential-socket\") pod \"0936eac3-9d11-4e29-a2a5-71de6af4424a\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " Jul 15 17:39:05.921080 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921055 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-certs\") pod \"0936eac3-9d11-4e29-a2a5-71de6af4424a\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " Jul 15 17:39:05.921403 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921088 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-data\") pod \"0936eac3-9d11-4e29-a2a5-71de6af4424a\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " Jul 15 17:39:05.921403 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921115 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-envoy\") pod \"0936eac3-9d11-4e29-a2a5-71de6af4424a\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " Jul 15 17:39:05.921403 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921146 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0936eac3-9d11-4e29-a2a5-71de6af4424a-istiod-ca-cert\") pod \"0936eac3-9d11-4e29-a2a5-71de6af4424a\" (UID: \"0936eac3-9d11-4e29-a2a5-71de6af4424a\") " Jul 15 17:39:05.921403 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921331 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0936eac3-9d11-4e29-a2a5-71de6af4424a" (UID: "0936eac3-9d11-4e29-a2a5-71de6af4424a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:05.921403 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921348 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0936eac3-9d11-4e29-a2a5-71de6af4424a" (UID: "0936eac3-9d11-4e29-a2a5-71de6af4424a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:05.921403 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921363 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0936eac3-9d11-4e29-a2a5-71de6af4424a" (UID: "0936eac3-9d11-4e29-a2a5-71de6af4424a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:05.921764 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921460 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:05.921764 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921473 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:05.921764 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921484 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:05.921764 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.921672 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0936eac3-9d11-4e29-a2a5-71de6af4424a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0936eac3-9d11-4e29-a2a5-71de6af4424a" (UID: "0936eac3-9d11-4e29-a2a5-71de6af4424a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:39:05.922131 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.922100 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-data" (OuterVolumeSpecName: "istio-data") pod "0936eac3-9d11-4e29-a2a5-71de6af4424a" (UID: "0936eac3-9d11-4e29-a2a5-71de6af4424a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:05.923366 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.923337 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-token" (OuterVolumeSpecName: "istio-token") pod "0936eac3-9d11-4e29-a2a5-71de6af4424a" (UID: "0936eac3-9d11-4e29-a2a5-71de6af4424a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:39:05.923488 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.923366 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0936eac3-9d11-4e29-a2a5-71de6af4424a" (UID: "0936eac3-9d11-4e29-a2a5-71de6af4424a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:39:05.923555 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.923509 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0936eac3-9d11-4e29-a2a5-71de6af4424a" (UID: "0936eac3-9d11-4e29-a2a5-71de6af4424a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:05.923828 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:05.923808 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-kube-api-access-wlj6z" (OuterVolumeSpecName: "kube-api-access-wlj6z") pod "0936eac3-9d11-4e29-a2a5-71de6af4424a" (UID: "0936eac3-9d11-4e29-a2a5-71de6af4424a"). InnerVolumeSpecName "kube-api-access-wlj6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:39:06.022119 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.022076 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:06.022119 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.022111 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:06.022119 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.022120 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0936eac3-9d11-4e29-a2a5-71de6af4424a-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:06.022119 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.022128 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:06.022403 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.022137 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:06.022403 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.022145 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlj6z\" (UniqueName: \"kubernetes.io/projected/0936eac3-9d11-4e29-a2a5-71de6af4424a-kube-api-access-wlj6z\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:06.342121 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.342087 2566 generic.go:356] "Generic (PLEG): container finished" podID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerID="803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876" exitCode=0 Jul 15 17:39:06.342320 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.342135 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" Jul 15 17:39:06.342320 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.342167 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" event={"ID":"0936eac3-9d11-4e29-a2a5-71de6af4424a","Type":"ContainerDied","Data":"803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876"} Jul 15 17:39:06.342320 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.342226 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6" event={"ID":"0936eac3-9d11-4e29-a2a5-71de6af4424a","Type":"ContainerDied","Data":"9c18e4d931d02f830a36de767aa0a372b7fe60bd610bc4571c6accc34ec58c9e"} Jul 15 17:39:06.342320 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.342251 2566 scope.go:117] "RemoveContainer" containerID="803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876" Jul 15 17:39:06.351087 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.351069 2566 scope.go:117] "RemoveContainer" containerID="167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b" Jul 15 17:39:06.359901 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.359881 2566 scope.go:117] "RemoveContainer" containerID="803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876" Jul 15 17:39:06.360222 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:39:06.360200 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876\": container with ID starting with 803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876 not found: ID does not exist" containerID="803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876" Jul 15 17:39:06.360337 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.360234 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876"} err="failed to get container status \"803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876\": rpc error: code = NotFound desc = could not find container \"803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876\": container with ID starting with 803c360765d660894c78729b168f8351d69a9081b22c7b47abf2a22bd8d03876 not found: ID does not exist" Jul 15 17:39:06.360337 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.360253 2566 scope.go:117] "RemoveContainer" containerID="167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b" Jul 15 17:39:06.360501 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:39:06.360484 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b\": container with ID starting with 167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b not found: ID does not exist" containerID="167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b" Jul 15 17:39:06.360545 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.360504 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b"} err="failed to get container status \"167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b\": rpc error: code = NotFound desc = could not find container \"167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b\": container with ID starting with 167bfef4ee4c642116b73f8da57a664ce10719e5d359d00364f69c8cde995b0b not found: ID does not exist" Jul 15 17:39:06.361052 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.361034 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6"] Jul 15 17:39:06.364578 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:06.364555 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4dfdb7-openshift-default-6c75695dbf-6qtg6"] Jul 15 17:39:08.188055 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:08.188011 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" path="/var/lib/kubelet/pods/0936eac3-9d11-4e29-a2a5-71de6af4424a/volumes" Jul 15 17:39:14.688047 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:14.688003 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:39:26.721398 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:26.721315 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp"] Jul 15 17:39:26.721842 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:26.721527 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" containerID="cri-o://573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649" gracePeriod=30 Jul 15 17:39:27.671162 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.671131 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47"] Jul 15 17:39:27.671548 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.671533 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" Jul 15 17:39:27.671652 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.671549 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" Jul 15 17:39:27.671652 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.671564 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" Jul 15 17:39:27.671652 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.671570 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" Jul 15 17:39:27.671809 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.671671 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" Jul 15 17:39:27.674890 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.674870 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.677918 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.677895 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-0bc296-openshift-default-dockercfg-kj9nl\"" Jul 15 17:39:27.687584 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.687547 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47"] Jul 15 17:39:27.718889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.718348 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-credential-socket\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.718889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.718393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d4bf66-a754-417d-be9c-a49a192a9e80-istiod-ca-cert\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.718889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.718434 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-socket\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.718889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.718476 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-certs\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.718889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.718497 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-data\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.718889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.718525 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-podinfo\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.718889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.718548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-token\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.718889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.718595 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk686\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-kube-api-access-zk686\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.718889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.718667 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-envoy\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.819658 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.819622 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-credential-socket\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.819658 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.819658 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d4bf66-a754-417d-be9c-a49a192a9e80-istiod-ca-cert\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.819683 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-socket\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.819710 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-certs\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.819725 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-data\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.819746 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-podinfo\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.819860 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-token\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.819890 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk686\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-kube-api-access-zk686\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.819922 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-envoy\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.820105 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-credential-socket\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.820114 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-socket\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820191 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.820173 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-certs\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820574 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.820256 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-data\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.820649 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.820577 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d4bf66-a754-417d-be9c-a49a192a9e80-istiod-ca-cert\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.822530 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.822510 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-envoy\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.822739 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.822719 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-podinfo\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.829121 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.829095 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk686\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-kube-api-access-zk686\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.829121 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.829109 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-token\") pod \"e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:27.986236 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:27.986142 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:28.128672 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:28.128646 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47"] Jul 15 17:39:28.130709 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:39:28.130679 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d4bf66_a754_417d_be9c_a49a192a9e80.slice/crio-a461ef3337e31387f49297343a9161209d67f3b055af24af9025194f8651eee6 WatchSource:0}: Error finding container a461ef3337e31387f49297343a9161209d67f3b055af24af9025194f8651eee6: Status 404 returned error can't find the container with id a461ef3337e31387f49297343a9161209d67f3b055af24af9025194f8651eee6 Jul 15 17:39:28.132804 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:28.132771 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:28.132921 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:28.132850 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:28.132921 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:28.132894 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:28.448288 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:28.448254 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" event={"ID":"38d4bf66-a754-417d-be9c-a49a192a9e80","Type":"ContainerStarted","Data":"fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583"} Jul 15 17:39:28.448465 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:28.448316 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" event={"ID":"38d4bf66-a754-417d-be9c-a49a192a9e80","Type":"ContainerStarted","Data":"a461ef3337e31387f49297343a9161209d67f3b055af24af9025194f8651eee6"} Jul 15 17:39:28.470010 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:28.469958 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" podStartSLOduration=1.4699426629999999 podStartE2EDuration="1.469942663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:39:28.468667498 +0000 UTC m=+2462.782372392" watchObservedRunningTime="2026-07-15 17:39:28.469942663 +0000 UTC m=+2462.783647613" Jul 15 17:39:28.986750 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:28.986703 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:29.991809 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:29.991776 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:30.457380 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:30.457346 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:30.458443 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:30.458425 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:31.974221 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:31.974196 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:39:32.060944 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.060836 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-data\") pod \"86175ad4-281c-4a41-ba79-eab7c5abbecc\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " Jul 15 17:39:32.060944 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.060897 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-certs\") pod \"86175ad4-281c-4a41-ba79-eab7c5abbecc\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " Jul 15 17:39:32.061186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.060982 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-token\") pod \"86175ad4-281c-4a41-ba79-eab7c5abbecc\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " Jul 15 17:39:32.061186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061051 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-envoy\") pod \"86175ad4-281c-4a41-ba79-eab7c5abbecc\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " Jul 15 17:39:32.061186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061078 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-credential-socket\") pod \"86175ad4-281c-4a41-ba79-eab7c5abbecc\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " Jul 15 17:39:32.061186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061138 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-socket\") pod \"86175ad4-281c-4a41-ba79-eab7c5abbecc\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " Jul 15 17:39:32.061186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061183 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-podinfo\") pod \"86175ad4-281c-4a41-ba79-eab7c5abbecc\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " Jul 15 17:39:32.061439 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061225 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z9hvw\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-kube-api-access-z9hvw\") pod \"86175ad4-281c-4a41-ba79-eab7c5abbecc\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " Jul 15 17:39:32.061439 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061255 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86175ad4-281c-4a41-ba79-eab7c5abbecc-istiod-ca-cert\") pod \"86175ad4-281c-4a41-ba79-eab7c5abbecc\" (UID: \"86175ad4-281c-4a41-ba79-eab7c5abbecc\") " Jul 15 17:39:32.061439 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061259 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "86175ad4-281c-4a41-ba79-eab7c5abbecc" (UID: "86175ad4-281c-4a41-ba79-eab7c5abbecc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:32.061595 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061516 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "86175ad4-281c-4a41-ba79-eab7c5abbecc" (UID: "86175ad4-281c-4a41-ba79-eab7c5abbecc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:32.061595 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061594 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:32.061753 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061633 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:32.061932 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061903 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-data" (OuterVolumeSpecName: "istio-data") pod "86175ad4-281c-4a41-ba79-eab7c5abbecc" (UID: "86175ad4-281c-4a41-ba79-eab7c5abbecc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:32.061932 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061912 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "86175ad4-281c-4a41-ba79-eab7c5abbecc" (UID: "86175ad4-281c-4a41-ba79-eab7c5abbecc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:32.062049 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.061947 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86175ad4-281c-4a41-ba79-eab7c5abbecc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "86175ad4-281c-4a41-ba79-eab7c5abbecc" (UID: "86175ad4-281c-4a41-ba79-eab7c5abbecc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:39:32.063969 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.063932 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "86175ad4-281c-4a41-ba79-eab7c5abbecc" (UID: "86175ad4-281c-4a41-ba79-eab7c5abbecc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:39:32.064143 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.064117 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "86175ad4-281c-4a41-ba79-eab7c5abbecc" (UID: "86175ad4-281c-4a41-ba79-eab7c5abbecc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:39:32.064259 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.064227 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-token" (OuterVolumeSpecName: "istio-token") pod "86175ad4-281c-4a41-ba79-eab7c5abbecc" (UID: "86175ad4-281c-4a41-ba79-eab7c5abbecc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:39:32.064518 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.064489 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-kube-api-access-z9hvw" (OuterVolumeSpecName: "kube-api-access-z9hvw") pod "86175ad4-281c-4a41-ba79-eab7c5abbecc" (UID: "86175ad4-281c-4a41-ba79-eab7c5abbecc"). InnerVolumeSpecName "kube-api-access-z9hvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:39:32.162134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.162098 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:32.162134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.162126 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:32.162134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.162137 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:32.162374 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.162147 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:32.162374 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.162157 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z9hvw\" (UniqueName: \"kubernetes.io/projected/86175ad4-281c-4a41-ba79-eab7c5abbecc-kube-api-access-z9hvw\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:32.162374 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.162166 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86175ad4-281c-4a41-ba79-eab7c5abbecc-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:32.162374 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.162173 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86175ad4-281c-4a41-ba79-eab7c5abbecc-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:39:32.468996 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.468954 2566 generic.go:356] "Generic (PLEG): container finished" podID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerID="573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649" exitCode=0 Jul 15 17:39:32.469181 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.469007 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" Jul 15 17:39:32.469181 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.469046 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" event={"ID":"86175ad4-281c-4a41-ba79-eab7c5abbecc","Type":"ContainerDied","Data":"573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649"} Jul 15 17:39:32.469181 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.469108 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp" event={"ID":"86175ad4-281c-4a41-ba79-eab7c5abbecc","Type":"ContainerDied","Data":"3cbd5f0e8dd257861265b922f7eb9e08854b4f6f8d80dffcc2fc8bde53935631"} Jul 15 17:39:32.469181 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.469131 2566 scope.go:117] "RemoveContainer" containerID="573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649" Jul 15 17:39:32.478675 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.478656 2566 scope.go:117] "RemoveContainer" containerID="a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1" Jul 15 17:39:32.487439 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.487419 2566 scope.go:117] "RemoveContainer" containerID="573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649" Jul 15 17:39:32.487814 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:39:32.487790 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649\": container with ID starting with 573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649 not found: ID does not exist" containerID="573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649" Jul 15 17:39:32.487911 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.487824 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649"} err="failed to get container status \"573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649\": rpc error: code = NotFound desc = could not find container \"573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649\": container with ID starting with 573981f421dbe0a224e7422149d4c6b01bea6cc8119659612a895682bb5a2649 not found: ID does not exist" Jul 15 17:39:32.487911 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.487842 2566 scope.go:117] "RemoveContainer" containerID="a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1" Jul 15 17:39:32.488145 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:39:32.488122 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1\": container with ID starting with a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1 not found: ID does not exist" containerID="a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1" Jul 15 17:39:32.488186 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.488152 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1"} err="failed to get container status \"a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1\": rpc error: code = NotFound desc = could not find container \"a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1\": container with ID starting with a284770322c21984ac182190a27aa012bb1cd75dfffe69a8df2edc3dffce5ba1 not found: ID does not exist" Jul 15 17:39:32.489133 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.489115 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp"] Jul 15 17:39:32.493226 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:32.493202 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-feaa2b-openshift-default-7c86b58d8c-wnttp"] Jul 15 17:39:34.188652 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:34.188592 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" path="/var/lib/kubelet/pods/86175ad4-281c-4a41-ba79-eab7c5abbecc/volumes" Jul 15 17:39:34.698920 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:34.698883 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:39:35.621058 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.621017 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc"] Jul 15 17:39:35.621573 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.621558 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" Jul 15 17:39:35.621651 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.621576 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" Jul 15 17:39:35.621651 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.621588 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" Jul 15 17:39:35.621651 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.621596 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" Jul 15 17:39:35.621762 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.621730 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" Jul 15 17:39:35.621762 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.621744 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0936eac3-9d11-4e29-a2a5-71de6af4424a" containerName="istio-proxy" Jul 15 17:39:35.625580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.625554 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.628573 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.628548 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-fadd3e-openshift-default-dockercfg-6srb9\"" Jul 15 17:39:35.637552 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.637527 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc"] Jul 15 17:39:35.694679 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.694644 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-data\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.694679 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.694681 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2npsq\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-kube-api-access-2npsq\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.694913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.694725 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-envoy\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.694913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.694747 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-token\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.694913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.694789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istiod-ca-cert\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.694913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.694823 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-socket\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.694913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.694838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-credential-socket\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.694913 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.694853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-certs\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.695167 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.694975 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-podinfo\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.795496 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.795458 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-socket\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.795496 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.795493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-credential-socket\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.795784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.795519 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-certs\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.795784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.795582 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-podinfo\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.795784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.795662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-data\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.795784 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.795684 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2npsq\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-kube-api-access-2npsq\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.795993 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.795946 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-socket\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.796050 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.795983 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-certs\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.796050 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.796002 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-credential-socket\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.796050 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.796020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-envoy\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.796207 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.796047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-token\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.796207 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.796091 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istiod-ca-cert\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.796443 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.796418 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-data\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.796690 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.796672 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istiod-ca-cert\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.798322 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.798290 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-envoy\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.798436 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.798371 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-podinfo\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.804716 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.804693 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-token\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.804852 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.804717 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2npsq\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-kube-api-access-2npsq\") pod \"e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:35.937409 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:35.937306 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:36.120593 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:36.120569 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc"] Jul 15 17:39:36.122634 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:39:36.122584 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd4ded6e_9325_481c_81c5_e5c1e9169c35.slice/crio-a62190327916701bd5e114568e52413bb06a589928de32f4707d7c317abd1188 WatchSource:0}: Error finding container a62190327916701bd5e114568e52413bb06a589928de32f4707d7c317abd1188: Status 404 returned error can't find the container with id a62190327916701bd5e114568e52413bb06a589928de32f4707d7c317abd1188 Jul 15 17:39:36.124927 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:36.124890 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:36.125021 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:36.124960 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:36.125021 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:36.124987 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:36.493886 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:36.493794 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" event={"ID":"bd4ded6e-9325-481c-81c5-e5c1e9169c35","Type":"ContainerStarted","Data":"9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07"} Jul 15 17:39:36.493886 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:36.493845 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" event={"ID":"bd4ded6e-9325-481c-81c5-e5c1e9169c35","Type":"ContainerStarted","Data":"a62190327916701bd5e114568e52413bb06a589928de32f4707d7c317abd1188"} Jul 15 17:39:36.517961 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:36.517913 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" podStartSLOduration=1.5178975860000001 podStartE2EDuration="1.517897586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 17:39:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:39:36.514986734 +0000 UTC m=+2470.828691628" watchObservedRunningTime="2026-07-15 17:39:36.517897586 +0000 UTC m=+2470.831602477" Jul 15 17:39:36.938108 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:36.938074 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:37.943361 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:37.943328 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:38.503551 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:38.503518 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:38.504590 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:38.504571 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:43.531426 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:43.531391 2566 generic.go:356] "Generic (PLEG): container finished" podID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerID="fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583" exitCode=0 Jul 15 17:39:43.531870 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:43.531448 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" event={"ID":"38d4bf66-a754-417d-be9c-a49a192a9e80","Type":"ContainerDied","Data":"fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583"} Jul 15 17:39:43.531917 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:43.531893 2566 scope.go:117] "RemoveContainer" containerID="fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583" Jul 15 17:39:43.532961 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:43.532922 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:43.533078 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:43.533024 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:43.533078 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:43.533072 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:44.538115 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:44.538082 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" event={"ID":"38d4bf66-a754-417d-be9c-a49a192a9e80","Type":"ContainerStarted","Data":"514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9"} Jul 15 17:39:44.986844 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:44.986806 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:44.988190 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:44.988165 2566 patch_prober.go:28] interesting pod/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 17:39:44.988325 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:44.988217 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 17:39:45.457431 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:45.457387 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:45.486391 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:45.486352 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:39:45.986967 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:45.986913 2566 patch_prober.go:28] interesting pod/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 17:39:45.987478 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:45.986981 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 17:39:46.986888 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:46.986845 2566 patch_prober.go:28] interesting pod/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 17:39:46.987203 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:46.987163 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 17:39:47.987469 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:47.987424 2566 patch_prober.go:28] interesting pod/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 15 17:39:47.988134 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:47.988099 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 15 17:39:48.990448 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:48.990418 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:48.991431 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:48.991413 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:39:53.742535 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:53.742223 2566 patch_prober.go:28] interesting pod/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:43494->10.132.0.51:15021: read: connection reset by peer" start-of-body= Jul 15 17:39:53.742535 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:53.742308 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:43494->10.132.0.51:15021: read: connection reset by peer" Jul 15 17:39:54.595574 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:54.595536 2566 generic.go:356] "Generic (PLEG): container finished" podID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerID="9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07" exitCode=0 Jul 15 17:39:54.595779 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:54.595638 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" event={"ID":"bd4ded6e-9325-481c-81c5-e5c1e9169c35","Type":"ContainerDied","Data":"9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07"} Jul 15 17:39:54.596062 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:54.596048 2566 scope.go:117] "RemoveContainer" containerID="9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07" Jul 15 17:39:54.597092 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:54.597062 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:54.597208 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:54.597148 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:54.597208 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:54.597195 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 15 17:39:55.602648 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:55.602591 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" event={"ID":"bd4ded6e-9325-481c-81c5-e5c1e9169c35","Type":"ContainerStarted","Data":"4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410"} Jul 15 17:39:55.938344 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:55.938251 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:55.938536 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:55.938511 2566 patch_prober.go:28] interesting pod/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 17:39:55.938588 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:55.938568 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 17:39:56.938814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:56.938777 2566 patch_prober.go:28] interesting pod/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 17:39:56.939214 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:56.938832 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 17:39:57.938411 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:57.938374 2566 patch_prober.go:28] interesting pod/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 17:39:57.938593 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:57.938434 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 17:39:58.194844 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:58.194762 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:39:58.938773 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:58.938734 2566 patch_prober.go:28] interesting pod/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 17:39:58.938959 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:58.938810 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 17:39:59.941777 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:59.941743 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:59.942243 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:59.942015 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:39:59.942864 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:39:59.942838 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:40:14.299330 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:40:14.299286 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:40:34.395129 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:40:34.395092 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:40:48.400040 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:40:48.399995 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:41:07.184889 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:41:07.184853 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:41:23.496104 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:41:23.496062 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:41:40.701362 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:41:40.701324 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:41:57.797282 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:41:57.797194 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:42:15.495501 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:42:15.495462 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:42:30.598423 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:42:30.598384 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:42:51.190744 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:42:51.190706 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:43:05.193905 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:43:05.193864 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:43:24.496442 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:43:24.496407 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:43:26.390456 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:43:26.390430 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:43:26.397209 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:43:26.397187 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:43:26.409066 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:43:26.409031 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:43:26.415397 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:43:26.415377 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-wb5hg_027a10e2-2eaa-42fa-8d0c-ceb61f74a71d/ovn-acl-logging/0.log" Jul 15 17:43:43.696387 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:43:43.696343 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:44:10.696806 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:44:10.696766 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:44:33.885214 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:44:33.885163 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:44:48.795840 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:44:48.795803 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:45:10.403767 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:45:10.403725 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:45:25.801596 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:45:25.801557 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:45:47.303400 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:45:47.303359 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:45:59.667708 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:45:59.667672 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47"] Jul 15 17:45:59.668274 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:45:59.667962 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" containerID="cri-o://514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9" gracePeriod=30 Jul 15 17:46:03.993485 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:03.993451 2566 patch_prober.go:28] interesting pod/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 17:46:03.993895 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:03.993504 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 17:46:04.937429 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.937359 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:46:04.975625 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.975580 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-token\") pod \"38d4bf66-a754-417d-be9c-a49a192a9e80\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " Jul 15 17:46:04.975814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.975635 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-envoy\") pod \"38d4bf66-a754-417d-be9c-a49a192a9e80\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " Jul 15 17:46:04.975814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.975664 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-socket\") pod \"38d4bf66-a754-417d-be9c-a49a192a9e80\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " Jul 15 17:46:04.975814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.975687 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-certs\") pod \"38d4bf66-a754-417d-be9c-a49a192a9e80\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " Jul 15 17:46:04.975814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.975754 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-data\") pod \"38d4bf66-a754-417d-be9c-a49a192a9e80\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " Jul 15 17:46:04.975814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.975778 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-credential-socket\") pod \"38d4bf66-a754-417d-be9c-a49a192a9e80\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " Jul 15 17:46:04.975814 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.975809 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk686\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-kube-api-access-zk686\") pod \"38d4bf66-a754-417d-be9c-a49a192a9e80\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " Jul 15 17:46:04.976109 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.975857 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-podinfo\") pod \"38d4bf66-a754-417d-be9c-a49a192a9e80\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " Jul 15 17:46:04.976109 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.975888 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d4bf66-a754-417d-be9c-a49a192a9e80-istiod-ca-cert\") pod \"38d4bf66-a754-417d-be9c-a49a192a9e80\" (UID: \"38d4bf66-a754-417d-be9c-a49a192a9e80\") " Jul 15 17:46:04.976109 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.976025 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38d4bf66-a754-417d-be9c-a49a192a9e80" (UID: "38d4bf66-a754-417d-be9c-a49a192a9e80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:04.976323 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.976267 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:04.976448 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.976406 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38d4bf66-a754-417d-be9c-a49a192a9e80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38d4bf66-a754-417d-be9c-a49a192a9e80" (UID: "38d4bf66-a754-417d-be9c-a49a192a9e80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:46:04.976653 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.976527 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38d4bf66-a754-417d-be9c-a49a192a9e80" (UID: "38d4bf66-a754-417d-be9c-a49a192a9e80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:04.976981 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.976953 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-data" (OuterVolumeSpecName: "istio-data") pod "38d4bf66-a754-417d-be9c-a49a192a9e80" (UID: "38d4bf66-a754-417d-be9c-a49a192a9e80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:04.977150 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.977128 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38d4bf66-a754-417d-be9c-a49a192a9e80" (UID: "38d4bf66-a754-417d-be9c-a49a192a9e80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:04.978215 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.978189 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-token" (OuterVolumeSpecName: "istio-token") pod "38d4bf66-a754-417d-be9c-a49a192a9e80" (UID: "38d4bf66-a754-417d-be9c-a49a192a9e80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:04.978389 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.978366 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-kube-api-access-zk686" (OuterVolumeSpecName: "kube-api-access-zk686") pod "38d4bf66-a754-417d-be9c-a49a192a9e80" (UID: "38d4bf66-a754-417d-be9c-a49a192a9e80"). InnerVolumeSpecName "kube-api-access-zk686". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:04.978754 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.978727 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38d4bf66-a754-417d-be9c-a49a192a9e80" (UID: "38d4bf66-a754-417d-be9c-a49a192a9e80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:04.978754 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:04.978729 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38d4bf66-a754-417d-be9c-a49a192a9e80" (UID: "38d4bf66-a754-417d-be9c-a49a192a9e80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:46:05.077237 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.077199 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:05.077237 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.077231 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38d4bf66-a754-417d-be9c-a49a192a9e80-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:05.077237 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.077240 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:05.077237 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.077248 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:05.077756 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.077256 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:05.077756 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.077264 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:05.077756 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.077272 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38d4bf66-a754-417d-be9c-a49a192a9e80-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:05.077756 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.077283 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk686\" (UniqueName: \"kubernetes.io/projected/38d4bf66-a754-417d-be9c-a49a192a9e80-kube-api-access-zk686\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:05.419931 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.419895 2566 generic.go:356] "Generic (PLEG): container finished" podID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerID="514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9" exitCode=0 Jul 15 17:46:05.420140 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.419946 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" Jul 15 17:46:05.420140 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.419983 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" event={"ID":"38d4bf66-a754-417d-be9c-a49a192a9e80","Type":"ContainerDied","Data":"514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9"} Jul 15 17:46:05.420140 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.420049 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47" event={"ID":"38d4bf66-a754-417d-be9c-a49a192a9e80","Type":"ContainerDied","Data":"a461ef3337e31387f49297343a9161209d67f3b055af24af9025194f8651eee6"} Jul 15 17:46:05.420140 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.420079 2566 scope.go:117] "RemoveContainer" containerID="514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9" Jul 15 17:46:05.429762 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.429743 2566 scope.go:117] "RemoveContainer" containerID="fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583" Jul 15 17:46:05.438423 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.438400 2566 scope.go:117] "RemoveContainer" containerID="514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9" Jul 15 17:46:05.438777 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:46:05.438735 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9\": container with ID starting with 514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9 not found: ID does not exist" containerID="514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9" Jul 15 17:46:05.438869 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.438789 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9"} err="failed to get container status \"514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9\": rpc error: code = NotFound desc = could not find container \"514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9\": container with ID starting with 514f8de4eed1b8fae419a4e8698fed44fb559a0fe9230b65cb7930eef14fd7c9 not found: ID does not exist" Jul 15 17:46:05.438869 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.438817 2566 scope.go:117] "RemoveContainer" containerID="fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583" Jul 15 17:46:05.439109 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:46:05.439094 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583\": container with ID starting with fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583 not found: ID does not exist" containerID="fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583" Jul 15 17:46:05.439144 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.439116 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583"} err="failed to get container status \"fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583\": rpc error: code = NotFound desc = could not find container \"fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583\": container with ID starting with fd2eef77c21b9a38c8b286726a5bc64b5527986e09fa5b0f43b1ef35cb1d0583 not found: ID does not exist" Jul 15 17:46:05.442762 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.442739 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47"] Jul 15 17:46:05.446687 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:05.446664 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-0bc296-openshift-default-7cf74555bc-wfj47"] Jul 15 17:46:06.190013 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:06.189969 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" path="/var/lib/kubelet/pods/38d4bf66-a754-417d-be9c-a49a192a9e80/volumes" Jul 15 17:46:13.690014 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:13.689976 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-z8fvg"] Jul 15 17:46:25.919909 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:25.919868 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc"] Jul 15 17:46:25.920366 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:25.920135 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" containerID="cri-o://4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410" gracePeriod=30 Jul 15 17:46:29.944900 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:29.944860 2566 patch_prober.go:28] interesting pod/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 17:46:29.945258 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:29.944935 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 17:46:31.179739 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.179678 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:46:31.325688 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.325646 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-socket\") pod \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " Jul 15 17:46:31.325688 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.325696 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-credential-socket\") pod \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " Jul 15 17:46:31.325944 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.325719 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-podinfo\") pod \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " Jul 15 17:46:31.325944 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.325737 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istiod-ca-cert\") pod \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " Jul 15 17:46:31.325944 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.325768 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-envoy\") pod \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " Jul 15 17:46:31.325944 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.325810 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-certs\") pod \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " Jul 15 17:46:31.325944 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.325858 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2npsq\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-kube-api-access-2npsq\") pod \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " Jul 15 17:46:31.325944 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.325917 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-data\") pod \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " Jul 15 17:46:31.325944 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.325939 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-token\") pod \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\" (UID: \"bd4ded6e-9325-481c-81c5-e5c1e9169c35\") " Jul 15 17:46:31.326304 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.326035 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd4ded6e-9325-481c-81c5-e5c1e9169c35" (UID: "bd4ded6e-9325-481c-81c5-e5c1e9169c35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:31.326304 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.326058 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd4ded6e-9325-481c-81c5-e5c1e9169c35" (UID: "bd4ded6e-9325-481c-81c5-e5c1e9169c35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:31.326304 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.326177 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd4ded6e-9325-481c-81c5-e5c1e9169c35" (UID: "bd4ded6e-9325-481c-81c5-e5c1e9169c35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:31.326477 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.326301 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd4ded6e-9325-481c-81c5-e5c1e9169c35" (UID: "bd4ded6e-9325-481c-81c5-e5c1e9169c35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 17:46:31.326477 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.326359 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:31.326477 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.326381 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-credential-socket\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:31.326477 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.326397 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-workload-certs\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:31.326918 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.326895 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd4ded6e-9325-481c-81c5-e5c1e9169c35" (UID: "bd4ded6e-9325-481c-81c5-e5c1e9169c35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:31.328090 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.328065 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd4ded6e-9325-481c-81c5-e5c1e9169c35" (UID: "bd4ded6e-9325-481c-81c5-e5c1e9169c35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 17:46:31.328296 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.328281 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd4ded6e-9325-481c-81c5-e5c1e9169c35" (UID: "bd4ded6e-9325-481c-81c5-e5c1e9169c35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:31.328597 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.328575 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd4ded6e-9325-481c-81c5-e5c1e9169c35" (UID: "bd4ded6e-9325-481c-81c5-e5c1e9169c35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 17:46:31.328692 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.328593 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-kube-api-access-2npsq" (OuterVolumeSpecName: "kube-api-access-2npsq") pod "bd4ded6e-9325-481c-81c5-e5c1e9169c35" (UID: "bd4ded6e-9325-481c-81c5-e5c1e9169c35"). InnerVolumeSpecName "kube-api-access-2npsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 17:46:31.427131 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.427092 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-envoy\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:31.427131 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.427125 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2npsq\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-kube-api-access-2npsq\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:31.427131 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.427135 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-data\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:31.427131 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.427145 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-token\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:31.427425 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.427153 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istio-podinfo\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:31.427425 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.427161 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd4ded6e-9325-481c-81c5-e5c1e9169c35-istiod-ca-cert\") on node \"ip-10-0-143-124.ec2.internal\" DevicePath \"\"" Jul 15 17:46:31.554189 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.554097 2566 generic.go:356] "Generic (PLEG): container finished" podID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerID="4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410" exitCode=0 Jul 15 17:46:31.554189 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.554139 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" event={"ID":"bd4ded6e-9325-481c-81c5-e5c1e9169c35","Type":"ContainerDied","Data":"4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410"} Jul 15 17:46:31.554189 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.554152 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" Jul 15 17:46:31.554189 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.554183 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc" event={"ID":"bd4ded6e-9325-481c-81c5-e5c1e9169c35","Type":"ContainerDied","Data":"a62190327916701bd5e114568e52413bb06a589928de32f4707d7c317abd1188"} Jul 15 17:46:31.554468 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.554207 2566 scope.go:117] "RemoveContainer" containerID="4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410" Jul 15 17:46:31.563686 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.563666 2566 scope.go:117] "RemoveContainer" containerID="9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07" Jul 15 17:46:31.571849 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.571827 2566 scope.go:117] "RemoveContainer" containerID="4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410" Jul 15 17:46:31.572114 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:46:31.572093 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410\": container with ID starting with 4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410 not found: ID does not exist" containerID="4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410" Jul 15 17:46:31.572164 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.572123 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410"} err="failed to get container status \"4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410\": rpc error: code = NotFound desc = could not find container \"4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410\": container with ID starting with 4b14a5bbbab22bba9c2303c0d639b1fa9ca4af5be54574d6043fc37009ea5410 not found: ID does not exist" Jul 15 17:46:31.572164 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.572141 2566 scope.go:117] "RemoveContainer" containerID="9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07" Jul 15 17:46:31.572368 ip-10-0-143-124 kubenswrapper[2566]: E0715 17:46:31.572352 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07\": container with ID starting with 9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07 not found: ID does not exist" containerID="9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07" Jul 15 17:46:31.572409 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.572371 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07"} err="failed to get container status \"9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07\": rpc error: code = NotFound desc = could not find container \"9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07\": container with ID starting with 9fac2d21a1337409d358ddff037d3e938ce30ea34d3811934c8645a6a7724b07 not found: ID does not exist" Jul 15 17:46:31.577149 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.577126 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc"] Jul 15 17:46:31.581552 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:31.581530 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-fadd3e-openshift-default-5f857cfd5f-dzmsc"] Jul 15 17:46:32.190985 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:32.190955 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" path="/var/lib/kubelet/pods/bd4ded6e-9325-481c-81c5-e5c1e9169c35/volumes" Jul 15 17:46:32.515964 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:32.515884 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ddh8f_f8c94165-df4d-4ece-8321-dbc69b172736/manager/0.log" Jul 15 17:46:32.770371 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:32.770300 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fkr6t_d4b3c857-5833-43d7-8a0b-eedcccc6f10c/server/0.log" Jul 15 17:46:32.895910 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:32.895883 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-w9q9k_4cb130c2-cb80-4489-8830-7bc57842431b/manager/0.log" Jul 15 17:46:33.021211 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:33.021130 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fdf764586-98m57_29c867b6-c059-4d03-a672-5a5b3448f24a/copy-manifests/0.log" Jul 15 17:46:33.029456 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:33.029436 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fdf764586-98m57_29c867b6-c059-4d03-a672-5a5b3448f24a/manager/0.log" Jul 15 17:46:37.592892 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:37.592862 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-z8fvg_75bc0d06-0fa6-40f9-9c85-e5a399d8b232/limitador/0.log" Jul 15 17:46:38.077228 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:38.077202 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786jxjh_0739cb2b-1aec-41a8-ba72-bbc05e29579c/istio-proxy/0.log" Jul 15 17:46:38.204938 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:38.204908 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-f6lt9_e112279e-c309-4462-945b-a91fc9faa1b3/discovery/0.log" Jul 15 17:46:38.441904 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:38.441819 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-667d575688-szp56_ecdaec29-7580-4ff3-a9a9-b2f122f75ebc/kube-auth-proxy/0.log" Jul 15 17:46:38.570388 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:38.570351 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bx4k7_9ab5193b-e598-4539-926d-007d5cec7ac5/istio-proxy/1.log" Jul 15 17:46:38.957254 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:38.957228 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d99b5d79b-cfd7r_9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1/router/0.log" Jul 15 17:46:39.787970 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:39.787934 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-hck2k_3d3372e5-1cb5-4e7a-8d6b-a7670561cb84/main/0.log" Jul 15 17:46:48.534895 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:48.534861 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d48rf_7c43931a-91df-49ea-bea3-10e907fe1b5c/global-pull-secret-syncer/0.log" Jul 15 17:46:48.769482 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:48.769443 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w9zmt_56812b13-8b82-41aa-83a9-0f5cf373ddfa/konnectivity-agent/0.log" Jul 15 17:46:48.841941 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:48.841910 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-124.ec2.internal_f0fb8ca27f1fff44d36146fb73f1ead9/haproxy/0.log" Jul 15 17:46:54.683892 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:54.683860 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-z8fvg_75bc0d06-0fa6-40f9-9c85-e5a399d8b232/limitador/0.log" Jul 15 17:46:56.496159 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:56.496127 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-wd9bp_8c9266b8-1422-4e7a-9736-89316363fc85/cluster-monitoring-operator/0.log" Jul 15 17:46:56.654670 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:56.654634 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f9jw8_11a16dca-6c39-4e50-9313-cda811d5395e/node-exporter/0.log" Jul 15 17:46:56.675690 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:56.675656 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f9jw8_11a16dca-6c39-4e50-9313-cda811d5395e/kube-rbac-proxy/0.log" Jul 15 17:46:56.696119 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:56.696090 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f9jw8_11a16dca-6c39-4e50-9313-cda811d5395e/init-textfile/0.log" Jul 15 17:46:58.540528 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.540502 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-76t88_06f2e5cb-6674-46b8-919f-1b9b9b578683/networking-console-plugin/0.log" Jul 15 17:46:58.818199 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818097 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds"] Jul 15 17:46:58.818684 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818655 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" Jul 15 17:46:58.818684 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818683 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" Jul 15 17:46:58.818875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818702 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" Jul 15 17:46:58.818875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818710 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" Jul 15 17:46:58.818875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818742 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" Jul 15 17:46:58.818875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818748 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" Jul 15 17:46:58.818875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818817 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="86175ad4-281c-4a41-ba79-eab7c5abbecc" containerName="istio-proxy" Jul 15 17:46:58.818875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818827 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" Jul 15 17:46:58.818875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818836 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" Jul 15 17:46:58.818875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818842 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd4ded6e-9325-481c-81c5-e5c1e9169c35" containerName="istio-proxy" Jul 15 17:46:58.818875 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.818851 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="38d4bf66-a754-417d-be9c-a49a192a9e80" containerName="istio-proxy" Jul 15 17:46:58.821896 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.821876 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.824850 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.824828 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kwr2v\"/\"default-dockercfg-kt2q7\"" Jul 15 17:46:58.826514 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.826494 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwr2v\"/\"kube-root-ca.crt\"" Jul 15 17:46:58.826594 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.826502 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwr2v\"/\"openshift-service-ca.crt\"" Jul 15 17:46:58.833270 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.833244 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds"] Jul 15 17:46:58.877451 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.877415 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-proc\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.877653 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.877470 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-lib-modules\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.877653 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.877536 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-sys\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.877653 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.877567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-podres\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.877653 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.877582 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc46z\" (UniqueName: \"kubernetes.io/projected/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-kube-api-access-qc46z\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.978036 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.977988 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-sys\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.978242 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.978054 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-podres\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.978242 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.978079 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc46z\" (UniqueName: \"kubernetes.io/projected/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-kube-api-access-qc46z\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.978242 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.978122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-sys\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.978242 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.978137 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-proc\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.978242 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.978164 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-podres\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.978242 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.978181 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-proc\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.978242 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.978233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-lib-modules\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.978549 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.978389 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-lib-modules\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:58.987321 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:58.987296 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc46z\" (UniqueName: \"kubernetes.io/projected/2c0cdd20-c0f4-42e7-a3a7-ea393a67e868-kube-api-access-qc46z\") pod \"perf-node-gather-daemonset-m2jds\" (UID: \"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868\") " pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:59.132906 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:59.132872 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:59.202176 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:59.202147 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/1.log" Jul 15 17:46:59.208416 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:59.208389 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-bb8gz_4f3aa3b2-8da9-47a5-a23e-72974a650bac/console-operator/2.log" Jul 15 17:46:59.273580 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:59.273532 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds"] Jul 15 17:46:59.275928 ip-10-0-143-124 kubenswrapper[2566]: W0715 17:46:59.275896 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2c0cdd20_c0f4_42e7_a3a7_ea393a67e868.slice/crio-7a25d56edd12d6341fe40cfc1d25000be1f09d391e5eaabd6fa3bc4f5539dd81 WatchSource:0}: Error finding container 7a25d56edd12d6341fe40cfc1d25000be1f09d391e5eaabd6fa3bc4f5539dd81: Status 404 returned error can't find the container with id 7a25d56edd12d6341fe40cfc1d25000be1f09d391e5eaabd6fa3bc4f5539dd81 Jul 15 17:46:59.691262 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:59.691219 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" event={"ID":"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868","Type":"ContainerStarted","Data":"36ac10a473a357f54475c6e4078442dbd1495663f111e62838c6c61bd7cd1c0a"} Jul 15 17:46:59.691681 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:59.691282 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" event={"ID":"2c0cdd20-c0f4-42e7-a3a7-ea393a67e868","Type":"ContainerStarted","Data":"7a25d56edd12d6341fe40cfc1d25000be1f09d391e5eaabd6fa3bc4f5539dd81"} Jul 15 17:46:59.691681 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:59.691319 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:46:59.708698 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:59.708638 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" podStartSLOduration=1.708589145 podStartE2EDuration="1.708589145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:46:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:46:59.708001557 +0000 UTC m=+2914.021706450" watchObservedRunningTime="2026-07-15 17:46:59.708589145 +0000 UTC m=+2914.022294036" Jul 15 17:46:59.729925 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:46:59.729899 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-xd962_cdeee4e8-f5bd-466e-94da-e7934bab2620/download-server/0.log" Jul 15 17:47:00.295557 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:00.295527 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-gldzd_596e6e61-c5fe-449a-833e-861d4588e9fb/volume-data-source-validator/0.log" Jul 15 17:47:01.370757 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:01.370724 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jht7b_30da06ef-d2dd-4cdf-bcfa-02bf28378483/dns/0.log" Jul 15 17:47:01.390214 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:01.390187 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jht7b_30da06ef-d2dd-4cdf-bcfa-02bf28378483/kube-rbac-proxy/0.log" Jul 15 17:47:01.560948 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:01.560919 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t8xnk_2d7dc39f-d2fc-42e9-9338-decd2e825747/dns-node-resolver/0.log" Jul 15 17:47:02.095831 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:02.095804 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2pxcv_25eeab2a-5a16-4d82-a085-22f22a60c078/node-ca/0.log" Jul 15 17:47:03.058239 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:03.058209 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786jxjh_0739cb2b-1aec-41a8-ba72-bbc05e29579c/istio-proxy/0.log" Jul 15 17:47:03.178632 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:03.178575 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-f6lt9_e112279e-c309-4462-945b-a91fc9faa1b3/discovery/0.log" Jul 15 17:47:03.225584 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:03.225553 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-667d575688-szp56_ecdaec29-7580-4ff3-a9a9-b2f122f75ebc/kube-auth-proxy/0.log" Jul 15 17:47:03.254215 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:03.254184 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bx4k7_9ab5193b-e598-4539-926d-007d5cec7ac5/istio-proxy/0.log" Jul 15 17:47:03.282654 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:03.282621 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bx4k7_9ab5193b-e598-4539-926d-007d5cec7ac5/istio-proxy/1.log" Jul 15 17:47:03.388245 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:03.388215 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d99b5d79b-cfd7r_9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1/router/0.log" Jul 15 17:47:04.140734 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:04.140696 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v78pr_3ee02540-b301-416f-9ac2-41c5170668c4/serve-healthcheck-canary/0.log" Jul 15 17:47:04.636538 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:04.636506 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-nzcjg_63bf6178-a18e-4b9c-81bb-98a008e87fc8/insights-operator/0.log" Jul 15 17:47:04.637279 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:04.637261 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-nzcjg_63bf6178-a18e-4b9c-81bb-98a008e87fc8/insights-operator/1.log" Jul 15 17:47:04.660443 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:04.660410 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4nrmg_f96fcf3d-4d29-4a71-a191-cb0b609a1602/kube-rbac-proxy/0.log" Jul 15 17:47:04.682322 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:04.682295 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4nrmg_f96fcf3d-4d29-4a71-a191-cb0b609a1602/exporter/0.log" Jul 15 17:47:04.704983 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:04.704959 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4nrmg_f96fcf3d-4d29-4a71-a191-cb0b609a1602/extractor/0.log" Jul 15 17:47:05.707839 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:05.707811 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kwr2v/perf-node-gather-daemonset-m2jds" Jul 15 17:47:07.148679 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:07.148641 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ddh8f_f8c94165-df4d-4ece-8321-dbc69b172736/manager/0.log" Jul 15 17:47:07.352420 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:07.352395 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fkr6t_d4b3c857-5833-43d7-8a0b-eedcccc6f10c/server/0.log" Jul 15 17:47:07.427436 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:07.427352 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-w9q9k_4cb130c2-cb80-4489-8830-7bc57842431b/manager/0.log" Jul 15 17:47:07.453862 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:07.453830 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fdf764586-98m57_29c867b6-c059-4d03-a672-5a5b3448f24a/manager/0.log" Jul 15 17:47:07.474233 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:07.474209 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fdf764586-98m57_29c867b6-c059-4d03-a672-5a5b3448f24a/copy-manifests/0.log" Jul 15 17:47:09.062637 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:09.062593 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786jxjh_0739cb2b-1aec-41a8-ba72-bbc05e29579c/istio-proxy/0.log" Jul 15 17:47:09.167387 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:09.167299 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-f6lt9_e112279e-c309-4462-945b-a91fc9faa1b3/discovery/0.log" Jul 15 17:47:09.194482 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:09.194453 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-667d575688-szp56_ecdaec29-7580-4ff3-a9a9-b2f122f75ebc/kube-auth-proxy/0.log" Jul 15 17:47:09.214860 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:09.214828 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bx4k7_9ab5193b-e598-4539-926d-007d5cec7ac5/istio-proxy/0.log" Jul 15 17:47:09.241537 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:09.241513 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bx4k7_9ab5193b-e598-4539-926d-007d5cec7ac5/istio-proxy/1.log" Jul 15 17:47:09.313823 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:09.313789 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d99b5d79b-cfd7r_9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1/router/0.log" Jul 15 17:47:10.588278 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:10.588246 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb6786jxjh_0739cb2b-1aec-41a8-ba72-bbc05e29579c/istio-proxy/0.log" Jul 15 17:47:10.695788 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:10.695751 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-f6lt9_e112279e-c309-4462-945b-a91fc9faa1b3/discovery/0.log" Jul 15 17:47:10.729928 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:10.729852 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-667d575688-szp56_ecdaec29-7580-4ff3-a9a9-b2f122f75ebc/kube-auth-proxy/0.log" Jul 15 17:47:10.749159 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:10.749122 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bx4k7_9ab5193b-e598-4539-926d-007d5cec7ac5/istio-proxy/0.log" Jul 15 17:47:10.780218 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:10.780177 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-bx4k7_9ab5193b-e598-4539-926d-007d5cec7ac5/istio-proxy/1.log" Jul 15 17:47:10.860197 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:10.860164 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7d99b5d79b-cfd7r_9038fd2e-b72d-46fe-8ac6-94f3e0bae3a1/router/0.log" Jul 15 17:47:13.979650 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:13.979620 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-l9pf5_055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe/migrator/0.log" Jul 15 17:47:14.001067 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:14.001041 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-l9pf5_055b8ea2-ffc0-4a22-bee4-6ef4d9f9dbbe/graceful-termination/0.log" Jul 15 17:47:14.341104 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:14.341072 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-qq9xb_8a3ef658-21e3-409d-973a-ef9afbbb3b46/kube-storage-version-migrator-operator/1.log" Jul 15 17:47:14.341993 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:14.341962 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-qq9xb_8a3ef658-21e3-409d-973a-ef9afbbb3b46/kube-storage-version-migrator-operator/0.log" Jul 15 17:47:15.699581 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.699544 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxzc4_5d65117a-a7e7-4feb-9a0f-f05010d0c737/kube-multus-additional-cni-plugins/0.log" Jul 15 17:47:15.720740 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.720711 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxzc4_5d65117a-a7e7-4feb-9a0f-f05010d0c737/egress-router-binary-copy/0.log" Jul 15 17:47:15.740700 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.740667 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxzc4_5d65117a-a7e7-4feb-9a0f-f05010d0c737/cni-plugins/0.log" Jul 15 17:47:15.762369 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.762339 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxzc4_5d65117a-a7e7-4feb-9a0f-f05010d0c737/bond-cni-plugin/0.log" Jul 15 17:47:15.782867 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.782834 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxzc4_5d65117a-a7e7-4feb-9a0f-f05010d0c737/routeoverride-cni/0.log" Jul 15 17:47:15.808718 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.808686 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxzc4_5d65117a-a7e7-4feb-9a0f-f05010d0c737/whereabouts-cni-bincopy/0.log" Jul 15 17:47:15.828413 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.828372 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mxzc4_5d65117a-a7e7-4feb-9a0f-f05010d0c737/whereabouts-cni/0.log" Jul 15 17:47:15.860613 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.860583 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lbkks_aaa5cd4e-6428-4ee2-9da8-fa0b5e24c5e7/kube-multus/0.log" Jul 15 17:47:15.973125 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.973036 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w7n7w_1d2712c7-d910-4a49-900d-ae83e771b3c1/network-metrics-daemon/0.log" Jul 15 17:47:15.992959 ip-10-0-143-124 kubenswrapper[2566]: I0715 17:47:15.992931 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w7n7w_1d2712c7-d910-4a49-900d-ae83e771b3c1/kube-rbac-proxy/0.log"