Jul 02 21:31:04.419090 ip-10-0-137-35 systemd[1]: Starting Kubernetes Kubelet... Jul 02 21:31:04.729791 ip-10-0-137-35 kubenswrapper[2547]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 21:31:04.729791 ip-10-0-137-35 kubenswrapper[2547]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 21:31:04.729791 ip-10-0-137-35 kubenswrapper[2547]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 21:31:04.729791 ip-10-0-137-35 kubenswrapper[2547]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 21:31:04.729791 ip-10-0-137-35 kubenswrapper[2547]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 21:31:04.731233 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.731151 2547 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 21:31:04.735015 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.734999 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:31:04.735015 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735015 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735019 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735022 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735027 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735030 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735033 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735038 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735042 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735045 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735048 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735051 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735054 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735058 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735060 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735063 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735066 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735069 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735072 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735075 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:31:04.735101 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735078 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735081 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735083 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735086 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735088 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735091 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735093 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735096 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735099 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735101 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735104 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735107 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735110 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735112 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735115 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735118 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735121 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735123 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735126 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735129 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:31:04.735559 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735131 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735135 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735139 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735141 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735145 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735148 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735151 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735154 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735157 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735159 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735162 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735165 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735168 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735170 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735173 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735176 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735179 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735182 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735185 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:31:04.736078 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735187 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735190 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735192 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735195 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735198 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735200 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735203 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735205 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735208 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735210 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735213 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735215 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735219 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735222 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735224 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735227 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735230 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735232 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735235 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735238 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:31:04.736542 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735240 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735242 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735246 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735248 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735251 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735253 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735256 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735260 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735263 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735265 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735268 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735270 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735273 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735275 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735278 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735280 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735283 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735285 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735288 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.735290 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:31:04.737031 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736004 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736011 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736014 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736017 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736019 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736022 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736025 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736029 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736032 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736035 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736037 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736040 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736042 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736045 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736047 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736050 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736053 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736055 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736058 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736061 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:31:04.737538 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736064 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736066 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736069 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736071 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736074 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736076 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736079 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736081 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736084 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736086 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736089 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736092 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736095 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736097 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736099 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736102 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736104 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736107 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736111 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736113 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:31:04.738045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736116 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736119 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736121 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736123 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736126 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736128 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736131 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736134 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736136 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736140 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736143 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736147 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736150 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736153 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736155 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736158 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736161 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736163 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736166 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736168 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:31:04.738580 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736171 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736173 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736176 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736179 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736181 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736183 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736188 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736191 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736194 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736197 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736200 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736210 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736213 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736217 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736220 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736222 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736225 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736228 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736230 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:31:04.739089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736233 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736236 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736238 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736241 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736243 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736246 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736248 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736251 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736253 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736256 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736258 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736261 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736263 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736265 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736269 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736273 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736276 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736279 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736281 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:31:04.739590 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736284 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736654 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736659 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736662 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736664 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736668 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736670 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736673 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736675 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736678 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736681 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736684 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736689 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736692 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736695 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736698 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736701 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736704 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736707 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:31:04.740065 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736709 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736712 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736715 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736717 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736720 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736723 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736725 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736728 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736731 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736734 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736736 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736739 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736741 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736743 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736747 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736750 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736753 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736757 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736759 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:31:04.740532 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736770 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736773 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736776 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736778 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736781 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736783 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736786 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736788 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736790 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736793 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736795 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736798 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736800 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736803 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736805 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736808 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736810 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736812 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736815 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736817 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:31:04.741111 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736833 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736836 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736838 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736840 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736843 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736845 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736848 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736850 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736854 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736856 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736859 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736862 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736865 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736874 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736877 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736880 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736883 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736885 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736888 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736891 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:31:04.741600 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736893 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736895 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736898 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736901 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736904 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736906 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736909 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736912 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736914 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736917 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736919 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736922 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736925 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736928 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736930 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736932 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736935 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736937 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736940 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736943 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:31:04.742098 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736945 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.736948 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737016 2547 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737023 2547 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737029 2547 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737033 2547 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737054 2547 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737057 2547 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737061 2547 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737066 2547 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737069 2547 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737072 2547 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737075 2547 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737079 2547 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737082 2547 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737085 2547 flags.go:64] FLAG: --cgroup-root="" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737088 2547 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737091 2547 flags.go:64] FLAG: --client-ca-file="" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737094 2547 flags.go:64] FLAG: --cloud-provider="external" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737097 2547 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737105 2547 flags.go:64] FLAG: --cluster-domain="" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737108 2547 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737111 2547 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 21:31:04.742576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737114 2547 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737117 2547 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737121 2547 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737124 2547 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737127 2547 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737131 2547 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737134 2547 flags.go:64] FLAG: --contention-profiling="false" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737137 2547 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737140 2547 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737143 2547 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737146 2547 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737151 2547 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737154 2547 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737157 2547 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737160 2547 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737163 2547 flags.go:64] FLAG: --enable-server="true" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737167 2547 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737171 2547 flags.go:64] FLAG: --event-burst="100" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737174 2547 flags.go:64] FLAG: --event-qps="50" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737177 2547 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737180 2547 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737183 2547 flags.go:64] FLAG: --eviction-hard="" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737187 2547 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737190 2547 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737193 2547 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 21:31:04.743132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737196 2547 flags.go:64] FLAG: --eviction-soft="" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737199 2547 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737202 2547 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737204 2547 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737207 2547 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737210 2547 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737213 2547 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737215 2547 flags.go:64] FLAG: --feature-gates="" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737219 2547 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737222 2547 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737226 2547 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737228 2547 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737232 2547 flags.go:64] FLAG: --healthz-port="10248" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737235 2547 flags.go:64] FLAG: --help="false" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737240 2547 flags.go:64] FLAG: --hostname-override="ip-10-0-137-35.ec2.internal" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737245 2547 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737248 2547 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737250 2547 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737254 2547 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737257 2547 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737260 2547 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737263 2547 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737266 2547 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 21:31:04.743780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737268 2547 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737272 2547 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737275 2547 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737278 2547 flags.go:64] FLAG: --kube-reserved="" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737280 2547 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737283 2547 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737286 2547 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737289 2547 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737292 2547 flags.go:64] FLAG: --lock-file="" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737294 2547 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737297 2547 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737300 2547 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737305 2547 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737308 2547 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737310 2547 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737313 2547 flags.go:64] FLAG: --logging-format="text" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737316 2547 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737320 2547 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737322 2547 flags.go:64] FLAG: --manifest-url="" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737325 2547 flags.go:64] FLAG: --manifest-url-header="" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737329 2547 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737332 2547 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737336 2547 flags.go:64] FLAG: --max-pods="110" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737340 2547 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737344 2547 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 21:31:04.744421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737347 2547 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737351 2547 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737354 2547 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737356 2547 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737359 2547 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737366 2547 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737369 2547 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737372 2547 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737376 2547 flags.go:64] FLAG: --pod-cidr="" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737379 2547 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737385 2547 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737388 2547 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737391 2547 flags.go:64] FLAG: --pods-per-core="0" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737394 2547 flags.go:64] FLAG: --port="10250" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737397 2547 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737399 2547 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01963dfc2be566be9" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737403 2547 flags.go:64] FLAG: --qos-reserved="" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737406 2547 flags.go:64] FLAG: --read-only-port="10255" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737409 2547 flags.go:64] FLAG: --register-node="true" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737412 2547 flags.go:64] FLAG: --register-with-taints="" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737415 2547 flags.go:64] FLAG: --registry-burst="10" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737418 2547 flags.go:64] FLAG: --registry-qps="5" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737420 2547 flags.go:64] FLAG: --reserved-cpus="" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737423 2547 flags.go:64] FLAG: --reserved-memory="" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737427 2547 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 21:31:04.745049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737430 2547 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737433 2547 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737435 2547 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737438 2547 flags.go:64] FLAG: --runonce="false" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737440 2547 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737445 2547 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737448 2547 flags.go:64] FLAG: --seccomp-default="false" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737452 2547 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737455 2547 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737458 2547 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737461 2547 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737464 2547 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737467 2547 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737469 2547 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737472 2547 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737475 2547 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737478 2547 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737481 2547 flags.go:64] FLAG: --system-cgroups="" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737484 2547 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737489 2547 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737492 2547 flags.go:64] FLAG: --tls-cert-file="" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737494 2547 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737499 2547 flags.go:64] FLAG: --tls-min-version="" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737502 2547 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737505 2547 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 21:31:04.745663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737508 2547 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737510 2547 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737513 2547 flags.go:64] FLAG: --v="2" Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737517 2547 flags.go:64] FLAG: --version="false" Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737520 2547 flags.go:64] FLAG: --vmodule="" Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737524 2547 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.737527 2547 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737639 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737643 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737646 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737648 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737651 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737655 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737658 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737661 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737664 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737668 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737671 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737674 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737677 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737680 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:31:04.746276 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737683 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737686 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737689 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737692 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737696 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737698 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737701 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737704 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737706 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737709 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737711 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737714 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737716 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737719 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737722 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737724 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737727 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737729 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737732 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737734 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:31:04.746797 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737737 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737739 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737742 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737746 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737748 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737754 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737757 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737760 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737763 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737767 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737770 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737773 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737776 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737779 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737781 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737784 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737787 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737791 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737794 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:31:04.747300 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737797 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737799 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737802 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737805 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737807 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737810 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737812 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737815 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737830 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737833 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737836 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737839 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737841 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737844 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737846 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737848 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737853 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737856 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737860 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737862 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:31:04.747810 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737865 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737868 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737870 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737873 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737876 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737878 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737881 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737890 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737892 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737895 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737897 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737901 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737903 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737906 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737908 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737911 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737913 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737916 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737918 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737921 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:31:04.748320 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737923 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737926 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737928 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737931 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737933 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.737936 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.738455 2547 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.744982 2547 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.744997 2547 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.745612 2547 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.745620 2547 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745676 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745682 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745685 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:31:04.748802 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745689 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745692 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745694 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745697 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745702 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745706 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745709 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745712 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745715 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745719 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745722 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745725 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745728 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745732 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745736 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745740 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745744 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745748 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745753 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:31:04.749184 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745756 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745759 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745762 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745765 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745768 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745770 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745773 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745777 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745779 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745782 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745785 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745788 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745793 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745800 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745804 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745808 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745812 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745815 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745833 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:31:04.749655 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745837 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745840 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745842 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745845 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745848 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745851 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745853 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745856 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745859 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745861 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745863 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745866 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745869 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745871 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745873 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745877 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745879 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745882 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745885 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745888 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:31:04.750139 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745891 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745894 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745897 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745899 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745902 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745904 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745906 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745909 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745911 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745914 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745917 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745919 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745921 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745924 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745926 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745929 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745931 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745934 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745937 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745940 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:31:04.750622 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745943 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745945 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745947 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745950 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745952 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745955 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745958 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745960 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745963 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745965 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745968 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745970 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745973 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745976 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745978 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745981 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745984 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:31:04.751122 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.745986 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.745992 2547 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746108 2547 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746112 2547 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746115 2547 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746118 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746121 2547 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746123 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746126 2547 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746129 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746132 2547 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746134 2547 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746137 2547 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746140 2547 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746143 2547 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 21:31:04.751554 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746146 2547 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746148 2547 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746151 2547 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746153 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746156 2547 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746159 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746161 2547 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746164 2547 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746166 2547 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746169 2547 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746171 2547 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746174 2547 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746176 2547 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746179 2547 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746182 2547 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746184 2547 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746186 2547 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746189 2547 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746191 2547 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746194 2547 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 21:31:04.751934 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746196 2547 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746199 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746201 2547 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746204 2547 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746208 2547 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746211 2547 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746213 2547 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746216 2547 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746219 2547 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746221 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746224 2547 feature_gate.go:328] unrecognized feature gate: Example Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746228 2547 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746231 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746234 2547 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746238 2547 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746241 2547 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746243 2547 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746246 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746249 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 21:31:04.752415 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746251 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746254 2547 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746257 2547 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746260 2547 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746262 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746265 2547 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746268 2547 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746270 2547 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746273 2547 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746276 2547 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746278 2547 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746281 2547 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746284 2547 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746286 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746289 2547 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746292 2547 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746295 2547 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746298 2547 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746300 2547 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746302 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 21:31:04.752895 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746305 2547 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746308 2547 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746311 2547 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746313 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746315 2547 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746319 2547 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746321 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746324 2547 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746326 2547 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746329 2547 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746331 2547 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746334 2547 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746336 2547 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746339 2547 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746342 2547 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746346 2547 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746348 2547 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746351 2547 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746353 2547 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 21:31:04.753382 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746356 2547 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746358 2547 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746361 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746363 2547 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746366 2547 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746368 2547 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746370 2547 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:04.746373 2547 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.746378 2547 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.746481 2547 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.748379 2547 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.749529 2547 server.go:1019] "Starting client certificate rotation" Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.749632 2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 21:31:04.753845 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.749664 2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 21:31:04.754168 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.751568 2547 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 21:31:04.757740 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.757719 2547 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 21:31:04.766813 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.766797 2547 log.go:25] "Validated CRI v1 runtime API" Jul 02 21:31:04.768799 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.768781 2547 log.go:25] "Validated CRI v1 image API" Jul 02 21:31:04.769583 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.769571 2547 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 21:31:04.773654 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.773630 2547 fs.go:135] Filesystem UUIDs: map[3edb2ae7-221f-4ae1-88ac-8204dd07b34e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 dfceb411-0b57-4750-9d4e-e8f844f7ca84:/dev/nvme0n1p3] Jul 02 21:31:04.773737 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.773651 2547 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 21:31:04.777976 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.777846 2547 manager.go:216] Machine: {Timestamp:2026-07-02 21:31:04.777352415 +0000 UTC m=+0.264747829 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111666 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28bf2a7b3e56426384405fce7d3bdb SystemUUID:ec28bf2a-7b3e-5642-6384-405fce7d3bdb BootID:3b8f8ff7-d74e-4296-a34a-c4dae8d4801c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a6:a9:59:8e:df Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a6:a9:59:8e:df Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:c5:29:fd:24:10 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 21:31:04.777976 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.777964 2547 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 21:31:04.778130 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.778063 2547 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 21:31:04.780438 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.780417 2547 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 21:31:04.780571 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.780441 2547 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-35.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 21:31:04.781027 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.781017 2547 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 21:31:04.781067 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.781029 2547 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 21:31:04.781067 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.781037 2547 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 21:31:04.781570 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.781560 2547 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 21:31:04.782717 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.782706 2547 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 21:31:04.785841 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.785803 2547 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 21:31:04.786331 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.786319 2547 state_mem.go:36] "Initialized new in-memory state store" Jul 02 21:31:04.786433 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.786424 2547 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 21:31:04.786480 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.786453 2547 kubelet.go:486] "Attempting to sync node with API server" Jul 02 21:31:04.786480 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.786463 2547 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 21:31:04.786480 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.786474 2547 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 21:31:04.786599 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.786484 2547 kubelet.go:398] "Adding apiserver pod source" Jul 02 21:31:04.786599 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.786492 2547 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 21:31:04.786853 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.786813 2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 21:31:04.786886 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.786866 2547 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 21:31:04.789129 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.789114 2547 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 21:31:04.790768 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.790754 2547 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 21:31:04.790850 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.790785 2547 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 21:31:04.791751 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791738 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 21:31:04.791786 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791755 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 21:31:04.791786 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791764 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 21:31:04.791786 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791785 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 21:31:04.791896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791794 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 21:31:04.791896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791800 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 21:31:04.791896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791815 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 21:31:04.791896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791837 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 21:31:04.791896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791846 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 21:31:04.791896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791852 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 21:31:04.791896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791867 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 21:31:04.791896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.791877 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 21:31:04.792464 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.792453 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 21:31:04.792464 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.792462 2547 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 21:31:04.795542 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.795527 2547 server.go:1268] "Started kubelet" Jul 02 21:31:04.795663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.795632 2547 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 21:31:04.795795 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.795776 2547 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 21:31:04.795861 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.795812 2547 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 21:31:04.796356 ip-10-0-137-35 systemd[1]: Started Kubernetes Kubelet. Jul 02 21:31:04.798692 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.798658 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 21:31:04.798953 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.798924 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 21:31:04.799032 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.798957 2547 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-35.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 21:31:04.799263 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.799242 2547 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 21:31:04.800112 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.800097 2547 server.go:310] "Adding debug handlers to kubelet server" Jul 02 21:31:04.801967 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.801946 2547 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 21:31:04.804903 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.804189 2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-35.ec2.internal.18be966569c257a4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-35.ec2.internal,UID:ip-10-0-137-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-35.ec2.internal,},FirstTimestamp:2026-07-02 21:31:04.7955025 +0000 UTC m=+0.282897913,LastTimestamp:2026-07-02 21:31:04.7955025 +0000 UTC m=+0.282897913,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-35.ec2.internal,}" Jul 02 21:31:04.805865 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.805844 2547 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 21:31:04.805865 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.805853 2547 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 21:31:04.806027 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.805954 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:04.806339 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.806313 2547 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 21:31:04.806339 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.806335 2547 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 21:31:04.806465 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.806315 2547 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 21:31:04.806465 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.806436 2547 reconstruct.go:97] "Volume reconstruction finished" Jul 02 21:31:04.806465 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.806442 2547 reconciler.go:29] "Reconciler: start to sync state" Jul 02 21:31:04.806811 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.806795 2547 factory.go:55] Registering systemd factory Jul 02 21:31:04.806811 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.806813 2547 factory.go:223] Registration of the systemd container factory successfully Jul 02 21:31:04.807065 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.807050 2547 factory.go:153] Registering CRI-O factory Jul 02 21:31:04.807065 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.807067 2547 factory.go:223] Registration of the crio container factory successfully Jul 02 21:31:04.807167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.807118 2547 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 21:31:04.807167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.807138 2547 factory.go:103] Registering Raw factory Jul 02 21:31:04.807167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.807151 2547 manager.go:1196] Started watching for new ooms in manager Jul 02 21:31:04.807573 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.807559 2547 manager.go:349] Starting recovery of all containers Jul 02 21:31:04.813995 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.813972 2547 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 02 21:31:04.813995 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.813989 2547 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 02 21:31:04.815767 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.815747 2547 manager.go:354] Recovery completed Jul 02 21:31:04.821355 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.821342 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:31:04.823236 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.823209 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:31:04.823286 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.823245 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:31:04.823286 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.823254 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:31:04.823800 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.823788 2547 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 21:31:04.823800 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.823796 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:31:04.823908 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.823799 2547 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 21:31:04.823952 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.823914 2547 state_mem.go:36] "Initialized new in-memory state store" Jul 02 21:31:04.824593 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.824567 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:31:04.824660 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.824598 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:31:04.824660 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.824613 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:31:04.825700 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.825547 2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-35.ec2.internal.18be96656b697710 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-35.ec2.internal,UID:ip-10-0-137-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-35.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-35.ec2.internal,},FirstTimestamp:2026-07-02 21:31:04.823232272 +0000 UTC m=+0.310627689,LastTimestamp:2026-07-02 21:31:04.823232272 +0000 UTC m=+0.310627689,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-35.ec2.internal,}" Jul 02 21:31:04.826182 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.826164 2547 policy_none.go:49] "None policy: Start" Jul 02 21:31:04.826264 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.826188 2547 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 21:31:04.826264 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.826211 2547 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 21:31:04.828416 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.828309 2547 policy_none.go:47] "Start" Jul 02 21:31:04.836638 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.836568 2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-35.ec2.internal.18be96656b69bbe6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-35.ec2.internal,UID:ip-10-0-137-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-35.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-35.ec2.internal,},FirstTimestamp:2026-07-02 21:31:04.823249894 +0000 UTC m=+0.310645307,LastTimestamp:2026-07-02 21:31:04.823249894 +0000 UTC m=+0.310645307,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-35.ec2.internal,}" Jul 02 21:31:04.837946 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.837931 2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wwrk" Jul 02 21:31:04.846252 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.846231 2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wwrk" Jul 02 21:31:04.847978 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.847918 2547 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-35.ec2.internal.18be96656b69dc92 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-35.ec2.internal,UID:ip-10-0-137-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-137-35.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-137-35.ec2.internal,},FirstTimestamp:2026-07-02 21:31:04.823258258 +0000 UTC m=+0.310653671,LastTimestamp:2026-07-02 21:31:04.823258258 +0000 UTC m=+0.310653671,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-35.ec2.internal,}" Jul 02 21:31:04.864380 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.864363 2547 manager.go:337] "Starting Device Plugin manager" Jul 02 21:31:04.864462 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.864393 2547 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 21:31:04.864462 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.864403 2547 server.go:87] "Starting device plugin registration server" Jul 02 21:31:04.864607 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.864595 2547 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 21:31:04.864642 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.864609 2547 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 21:31:04.864718 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.864696 2547 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 21:31:04.864793 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.864778 2547 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 21:31:04.864793 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.864791 2547 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 21:31:04.865364 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.865344 2547 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 21:31:04.865435 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.865392 2547 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:04.963126 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.963085 2547 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 21:31:04.965306 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.964217 2547 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 21:31:04.965306 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.964271 2547 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 21:31:04.965306 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.964305 2547 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 21:31:04.965306 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.964349 2547 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 21:31:04.965306 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.964746 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:31:04.965643 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.965623 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:31:04.965771 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.965662 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:31:04.965771 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.965680 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:31:04.965771 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.965710 2547 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-35.ec2.internal" Jul 02 21:31:04.966552 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.966536 2547 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:31:04.974797 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:04.974781 2547 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-35.ec2.internal" Jul 02 21:31:04.974908 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.974806 2547 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-35.ec2.internal\": node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:04.998879 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:04.998810 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:05.065414 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.065377 2547 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal"] Jul 02 21:31:05.065570 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.065471 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:31:05.068031 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.068015 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:31:05.068122 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.068050 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:31:05.068122 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.068060 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:31:05.070330 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.070316 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:31:05.070473 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.070458 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.070536 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.070498 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:31:05.072166 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.071282 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:31:05.072166 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.071312 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:31:05.072166 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.071327 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:31:05.072166 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.071607 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:31:05.072166 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.071624 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:31:05.072166 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.071635 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:31:05.073886 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.073870 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.073950 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.073897 2547 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 21:31:05.074588 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.074566 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientMemory" Jul 02 21:31:05.074674 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.074595 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 21:31:05.074674 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.074613 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeHasSufficientPID" Jul 02 21:31:05.099179 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.099144 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:05.104628 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.104612 2547 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-35.ec2.internal\" not found" node="ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.108082 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.108057 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47de5f2930588a3e744010cfb8dff070-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal\" (UID: \"47de5f2930588a3e744010cfb8dff070\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.108147 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.108098 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47de5f2930588a3e744010cfb8dff070-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal\" (UID: \"47de5f2930588a3e744010cfb8dff070\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.108147 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.108127 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c48cb01bf393d73b784190e0bda418c3-config\") pod \"kube-apiserver-proxy-ip-10-0-137-35.ec2.internal\" (UID: \"c48cb01bf393d73b784190e0bda418c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.109288 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.109272 2547 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-35.ec2.internal\" not found" node="ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.200005 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.199969 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:05.208302 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.208269 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c48cb01bf393d73b784190e0bda418c3-config\") pod \"kube-apiserver-proxy-ip-10-0-137-35.ec2.internal\" (UID: \"c48cb01bf393d73b784190e0bda418c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.208431 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.208314 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47de5f2930588a3e744010cfb8dff070-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal\" (UID: \"47de5f2930588a3e744010cfb8dff070\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.208431 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.208340 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47de5f2930588a3e744010cfb8dff070-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal\" (UID: \"47de5f2930588a3e744010cfb8dff070\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.208431 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.208379 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c48cb01bf393d73b784190e0bda418c3-config\") pod \"kube-apiserver-proxy-ip-10-0-137-35.ec2.internal\" (UID: \"c48cb01bf393d73b784190e0bda418c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.208431 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.208387 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47de5f2930588a3e744010cfb8dff070-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal\" (UID: \"47de5f2930588a3e744010cfb8dff070\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.208431 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.208392 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47de5f2930588a3e744010cfb8dff070-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal\" (UID: \"47de5f2930588a3e744010cfb8dff070\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.300768 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.300734 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:05.401226 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.401204 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:05.407462 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.407439 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.412590 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.412458 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.502223 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.502183 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:05.602846 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.602741 2547 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-35.ec2.internal\" not found" Jul 02 21:31:05.664429 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.664404 2547 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:31:05.706404 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.706377 2547 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.726953 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.726932 2547 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 21:31:05.728413 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.728397 2547 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal" Jul 02 21:31:05.737331 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.737310 2547 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 21:31:05.749497 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.749480 2547 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 21:31:05.749636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.749614 2547 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 21:31:05.749691 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.749631 2547 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 21:31:05.786680 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.786654 2547 apiserver.go:52] "Watching apiserver" Jul 02 21:31:05.802235 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.802010 2547 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 21:31:05.803414 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.803387 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-84htm","kube-system/global-pull-secret-syncer-hsd7b","kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7","openshift-cluster-node-tuning-operator/tuned-rj46c","openshift-dns/node-resolver-l2nsz","openshift-image-registry/node-ca-gjvt4","openshift-multus/multus-additional-cni-plugins-bswhr","openshift-multus/network-metrics-daemon-95lhp","kube-system/konnectivity-agent-4z64k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal","openshift-multus/multus-szhvf","openshift-network-diagnostics/network-check-target-pqqpn","openshift-network-operator/iptables-alerter-frwtg"] Jul 02 21:31:05.805945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.805924 2547 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 21:31:05.806114 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.806097 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.808705 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.808684 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:05.808799 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.808755 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:05.809037 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.809017 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 21:31:05.809268 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.809244 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 21:31:05.809268 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.809256 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 21:31:05.809443 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.809250 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 21:31:05.809443 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.809291 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jzdgc\"" Jul 02 21:31:05.809443 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.809332 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 21:31:05.809443 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.809249 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 21:31:05.811280 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.811264 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:05.812111 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812094 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-systemd-units\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812201 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812117 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-run-systemd\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812201 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812131 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-run-openvswitch\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812201 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812145 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-run-ovn\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812201 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812159 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-node-log\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812201 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812173 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-cni-bin\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812356 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812212 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/818e1edd-f50a-43b0-86a5-03c2c318e468-env-overrides\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812356 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812235 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:05.812356 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812262 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-kubelet\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812356 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812277 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-log-socket\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812356 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812292 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812356 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812314 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/818e1edd-f50a-43b0-86a5-03c2c318e468-ovnkube-config\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812356 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812335 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/818e1edd-f50a-43b0-86a5-03c2c318e468-ovn-node-metrics-cert\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812356 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812350 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e6c4320e-5777-4358-8dac-d43f1f4f411e-dbus\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:05.812661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812365 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-run-netns\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812379 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-var-lib-openvswitch\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812403 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-run-ovn-kubernetes\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812418 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-cni-netd\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812439 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/818e1edd-f50a-43b0-86a5-03c2c318e468-ovnkube-script-lib\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812454 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jwwm\" (UniqueName: \"kubernetes.io/projected/818e1edd-f50a-43b0-86a5-03c2c318e468-kube-api-access-4jwwm\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812473 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e6c4320e-5777-4358-8dac-d43f1f4f411e-kubelet-config\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:05.812661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812514 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-slash\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.812661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.812550 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-etc-openvswitch\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.813576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.813554 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dbzhs\"" Jul 02 21:31:05.813652 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.813572 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 21:31:05.813652 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.813581 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 21:31:05.813752 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.813666 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 21:31:05.814039 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.814023 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.816495 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.816475 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 21:31:05.816590 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.816566 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:31:05.816656 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.816591 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5tv9r\"" Jul 02 21:31:05.816724 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.816704 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:05.818950 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.818932 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fml4l\"" Jul 02 21:31:05.819058 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.818932 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 21:31:05.819058 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.819038 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 21:31:05.819332 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.819312 2547 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 21:31:05.819431 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.819351 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:05.821626 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.821606 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 21:31:05.821750 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.821702 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 21:31:05.821750 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.821702 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jtq2t\"" Jul 02 21:31:05.821947 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.821931 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 21:31:05.822275 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.822260 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:05.824676 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.824661 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 21:31:05.824922 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.824710 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 21:31:05.824922 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.824899 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:05.825067 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.825047 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 21:31:05.825124 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.825092 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:05.825124 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.825116 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 21:31:05.825446 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.825431 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-dxrws\"" Jul 02 21:31:05.825501 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.825481 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 21:31:05.828104 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.828086 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:05.831117 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.831092 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xbvgf\"" Jul 02 21:31:05.831230 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.831210 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 21:31:05.831305 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.831288 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 21:31:05.832973 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.832955 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.835397 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.835381 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:05.835546 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.835527 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:05.835624 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.835548 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 21:31:05.835624 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.835614 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pphch\"" Jul 02 21:31:05.837594 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.837578 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:05.839720 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.839703 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4tpdp\"" Jul 02 21:31:05.840078 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.840064 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:31:05.840161 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.840149 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 21:31:05.840320 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.840309 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 21:31:05.848745 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.848725 2547 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-47q7t" Jul 02 21:31:05.849126 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.849091 2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 21:26:05 +0000 UTC" deadline="2028-04-19 14:53:58.16838344 +0000 UTC" Jul 02 21:31:05.849126 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.849125 2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15761h22m52.319260368s" Jul 02 21:31:05.857406 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.857388 2547 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-47q7t" Jul 02 21:31:05.907763 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.907745 2547 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 21:31:05.913051 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913030 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-run-ovn-kubernetes\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913157 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913057 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-cni-netd\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913157 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913075 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jwwm\" (UniqueName: \"kubernetes.io/projected/818e1edd-f50a-43b0-86a5-03c2c318e468-kube-api-access-4jwwm\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913157 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913094 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-sysctl-d\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.913157 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913112 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/47e464a7-c6bb-45fa-89b9-a4c5eb701478-serviceca\") pod \"node-ca-gjvt4\" (UID: \"47e464a7-c6bb-45fa-89b9-a4c5eb701478\") " pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913157 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-system-cni-dir\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913166 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-cni-netd\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913195 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-run-openvswitch\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913217 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-run-ovn\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913236 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-cni-bin\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913194 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-run-ovn-kubernetes\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913259 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913268 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-run-ovn\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913273 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-run-openvswitch\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913284 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/524c6758-8675-43f8-8da0-80f32a490537-host-slash\") pod \"iptables-alerter-frwtg\" (UID: \"524c6758-8675-43f8-8da0-80f32a490537\") " pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913292 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-cni-bin\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913322 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-daemon-config\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.913344 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:05.913350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913356 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/818e1edd-f50a-43b0-86a5-03c2c318e468-ovn-node-metrics-cert\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.914005 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:05.913409 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret podName:e6c4320e-5777-4358-8dac-d43f1f4f411e nodeName:}" failed. No retries permitted until 2026-07-02 21:31:06.413385436 +0000 UTC m=+1.900780851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret") pod "global-pull-secret-syncer-hsd7b" (UID: "e6c4320e-5777-4358-8dac-d43f1f4f411e") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:05.914005 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913744 2547 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 21:31:05.914005 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913895 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-systemd\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.914005 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913932 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-run-netns\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.914005 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913957 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e6c4320e-5777-4358-8dac-d43f1f4f411e-kubelet-config\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:05.914005 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.913986 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-sys-fs\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:05.914005 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914002 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-run-netns\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914012 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6bcf0af6-b67a-4338-b2af-6431464934f1-agent-certs\") pod \"konnectivity-agent-4z64k\" (UID: \"6bcf0af6-b67a-4338-b2af-6431464934f1\") " pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914046 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/524c6758-8675-43f8-8da0-80f32a490537-iptables-alerter-script\") pod \"iptables-alerter-frwtg\" (UID: \"524c6758-8675-43f8-8da0-80f32a490537\") " pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914075 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-cnibin\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914077 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e6c4320e-5777-4358-8dac-d43f1f4f411e-kubelet-config\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914109 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-etc-openvswitch\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914131 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-device-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914159 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk9bs\" (UniqueName: \"kubernetes.io/projected/0a254df8-e862-4e27-bb25-3139b2de01d0-kube-api-access-zk9bs\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914182 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-kubernetes\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914211 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25b068f3-c4ca-4d18-b74c-1b8d653d27c8-hosts-file\") pod \"node-resolver-l2nsz\" (UID: \"25b068f3-c4ca-4d18-b74c-1b8d653d27c8\") " pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:05.914341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914237 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-etc-openvswitch\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.914850 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914712 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k8sn\" (UniqueName: \"kubernetes.io/projected/47e464a7-c6bb-45fa-89b9-a4c5eb701478-kube-api-access-6k8sn\") pod \"node-ca-gjvt4\" (UID: \"47e464a7-c6bb-45fa-89b9-a4c5eb701478\") " pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:05.914850 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914743 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-socket-dir-parent\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.914850 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914791 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-conf-dir\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.914850 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914840 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/818e1edd-f50a-43b0-86a5-03c2c318e468-env-overrides\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.915035 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914894 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b035aac5-e945-44dd-a4f6-131fca47ad48-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:05.915035 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.914952 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnt6d\" (UniqueName: \"kubernetes.io/projected/b035aac5-e945-44dd-a4f6-131fca47ad48-kube-api-access-lnt6d\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:05.915127 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915080 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndgdf\" (UniqueName: \"kubernetes.io/projected/d20d9255-7e0d-4fd4-a553-18bb3769366c-kube-api-access-ndgdf\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.915210 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915125 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:05.915365 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915344 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-kubelet\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.915434 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915421 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e6c4320e-5777-4358-8dac-d43f1f4f411e-dbus\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:05.915517 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915476 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:05.915731 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915710 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e6c4320e-5777-4358-8dac-d43f1f4f411e-dbus\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:05.915833 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915760 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25b068f3-c4ca-4d18-b74c-1b8d653d27c8-tmp-dir\") pod \"node-resolver-l2nsz\" (UID: \"25b068f3-c4ca-4d18-b74c-1b8d653d27c8\") " pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:05.915833 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915793 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-os-release\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.915943 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915861 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-var-lib-openvswitch\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.915943 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915919 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-sys\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.916046 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915952 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-lib-modules\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.916046 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915954 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-kubelet\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.916046 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.915995 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-var-lib-openvswitch\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.916191 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916031 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e60d724-3791-4386-84d2-f58f064342ef-etc-tuned\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.916191 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916098 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47e464a7-c6bb-45fa-89b9-a4c5eb701478-host\") pod \"node-ca-gjvt4\" (UID: \"47e464a7-c6bb-45fa-89b9-a4c5eb701478\") " pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:05.916191 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916130 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:05.916191 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916157 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-hostroot\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.916191 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916185 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-sysctl-conf\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.916429 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916210 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-run-k8s-cni-cncf-io\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.916429 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916231 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:05.916429 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916263 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-systemd-units\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.916429 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916293 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-run-systemd\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.916429 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916323 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-system-cni-dir\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:05.916429 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916352 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d20d9255-7e0d-4fd4-a553-18bb3769366c-cni-binary-copy\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.916429 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916378 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-etc-kubernetes\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.916429 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916411 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-log-socket\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916458 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/818e1edd-f50a-43b0-86a5-03c2c318e468-ovnkube-config\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916517 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916553 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcq2g\" (UniqueName: \"kubernetes.io/projected/f4f1ee0d-0554-4134-b502-08c8aefca7ec-kube-api-access-lcq2g\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916580 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-cnibin\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916613 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-run-netns\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916645 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-var-lib-kubelet\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916681 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/818e1edd-f50a-43b0-86a5-03c2c318e468-ovnkube-script-lib\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916712 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-registration-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916759 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6bcf0af6-b67a-4338-b2af-6431464934f1-konnectivity-ca\") pod \"konnectivity-agent-4z64k\" (UID: \"6bcf0af6-b67a-4338-b2af-6431464934f1\") " pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916792 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-log-socket\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.916929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916836 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-var-lib-kubelet\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.916982 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-systemd-units\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917030 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-run-systemd\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917215 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qlrs\" (UniqueName: \"kubernetes.io/projected/6e60d724-3791-4386-84d2-f58f064342ef-kube-api-access-5qlrs\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917245 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-os-release\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917275 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-var-lib-cni-multus\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917301 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-slash\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917330 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-socket-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917359 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b035aac5-e945-44dd-a4f6-131fca47ad48-cni-binary-copy\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917388 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-cni-dir\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917412 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-var-lib-cni-bin\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917442 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-node-log\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917470 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhd9l\" (UniqueName: \"kubernetes.io/projected/524c6758-8675-43f8-8da0-80f32a490537-kube-api-access-jhd9l\") pod \"iptables-alerter-frwtg\" (UID: \"524c6758-8675-43f8-8da0-80f32a490537\") " pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917501 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-sysconfig\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917528 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-run\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917556 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917586 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-modprobe-d\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.917636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917621 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-host\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917647 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e60d724-3791-4386-84d2-f58f064342ef-tmp\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917693 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcr5p\" (UniqueName: \"kubernetes.io/projected/25b068f3-c4ca-4d18-b74c-1b8d653d27c8-kube-api-access-tcr5p\") pod \"node-resolver-l2nsz\" (UID: \"25b068f3-c4ca-4d18-b74c-1b8d653d27c8\") " pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917720 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b035aac5-e945-44dd-a4f6-131fca47ad48-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917749 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-run-multus-certs\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917900 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-slash\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.917976 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-node-log\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.918043 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/818e1edd-f50a-43b0-86a5-03c2c318e468-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.918048 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/818e1edd-f50a-43b0-86a5-03c2c318e468-ovn-node-metrics-cert\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.918369 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/818e1edd-f50a-43b0-86a5-03c2c318e468-ovnkube-script-lib\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.918719 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/818e1edd-f50a-43b0-86a5-03c2c318e468-ovnkube-config\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.919838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.919608 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/818e1edd-f50a-43b0-86a5-03c2c318e468-env-overrides\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.921754 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.921732 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jwwm\" (UniqueName: \"kubernetes.io/projected/818e1edd-f50a-43b0-86a5-03c2c318e468-kube-api-access-4jwwm\") pod \"ovnkube-node-84htm\" (UID: \"818e1edd-f50a-43b0-86a5-03c2c318e468\") " pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:05.968951 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:05.968922 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc48cb01bf393d73b784190e0bda418c3.slice/crio-d0ff255fe6122bbe3102e1f7790d7d4b71e9fd6ebff317bab7637a100b358e07 WatchSource:0}: Error finding container d0ff255fe6122bbe3102e1f7790d7d4b71e9fd6ebff317bab7637a100b358e07: Status 404 returned error can't find the container with id d0ff255fe6122bbe3102e1f7790d7d4b71e9fd6ebff317bab7637a100b358e07 Jul 02 21:31:05.969530 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:05.969513 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47de5f2930588a3e744010cfb8dff070.slice/crio-ca2b9c85afaf5aa020f20f556d6930fc462ea58a8aa6de00ee6e9075f49d23a3 WatchSource:0}: Error finding container ca2b9c85afaf5aa020f20f556d6930fc462ea58a8aa6de00ee6e9075f49d23a3: Status 404 returned error can't find the container with id ca2b9c85afaf5aa020f20f556d6930fc462ea58a8aa6de00ee6e9075f49d23a3 Jul 02 21:31:05.975198 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:05.975182 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:31:06.018564 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018532 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-systemd\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.018564 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018567 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-sys-fs\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018589 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6bcf0af6-b67a-4338-b2af-6431464934f1-agent-certs\") pod \"konnectivity-agent-4z64k\" (UID: \"6bcf0af6-b67a-4338-b2af-6431464934f1\") " pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018606 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/524c6758-8675-43f8-8da0-80f32a490537-iptables-alerter-script\") pod \"iptables-alerter-frwtg\" (UID: \"524c6758-8675-43f8-8da0-80f32a490537\") " pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018625 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-cnibin\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018644 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-device-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018659 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-systemd\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018670 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk9bs\" (UniqueName: \"kubernetes.io/projected/0a254df8-e862-4e27-bb25-3139b2de01d0-kube-api-access-zk9bs\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018699 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-cnibin\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018669 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-sys-fs\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018723 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-kubernetes\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018740 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-device-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018768 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25b068f3-c4ca-4d18-b74c-1b8d653d27c8-hosts-file\") pod \"node-resolver-l2nsz\" (UID: \"25b068f3-c4ca-4d18-b74c-1b8d653d27c8\") " pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018782 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-kubernetes\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.018788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018793 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k8sn\" (UniqueName: \"kubernetes.io/projected/47e464a7-c6bb-45fa-89b9-a4c5eb701478-kube-api-access-6k8sn\") pod \"node-ca-gjvt4\" (UID: \"47e464a7-c6bb-45fa-89b9-a4c5eb701478\") " pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018839 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-socket-dir-parent\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018847 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25b068f3-c4ca-4d18-b74c-1b8d653d27c8-hosts-file\") pod \"node-resolver-l2nsz\" (UID: \"25b068f3-c4ca-4d18-b74c-1b8d653d27c8\") " pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018884 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-conf-dir\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018916 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b035aac5-e945-44dd-a4f6-131fca47ad48-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018908 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-socket-dir-parent\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018951 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnt6d\" (UniqueName: \"kubernetes.io/projected/b035aac5-e945-44dd-a4f6-131fca47ad48-kube-api-access-lnt6d\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018966 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-conf-dir\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.018987 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndgdf\" (UniqueName: \"kubernetes.io/projected/d20d9255-7e0d-4fd4-a553-18bb3769366c-kube-api-access-ndgdf\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019020 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019046 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019074 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25b068f3-c4ca-4d18-b74c-1b8d653d27c8-tmp-dir\") pod \"node-resolver-l2nsz\" (UID: \"25b068f3-c4ca-4d18-b74c-1b8d653d27c8\") " pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019098 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-os-release\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019121 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-sys\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019151 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-lib-modules\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019130 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019174 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e60d724-3791-4386-84d2-f58f064342ef-etc-tuned\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019197 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47e464a7-c6bb-45fa-89b9-a4c5eb701478-host\") pod \"node-ca-gjvt4\" (UID: \"47e464a7-c6bb-45fa-89b9-a4c5eb701478\") " pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:06.019398 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:06.019254 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019249 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019280 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/524c6758-8675-43f8-8da0-80f32a490537-iptables-alerter-script\") pod \"iptables-alerter-frwtg\" (UID: \"524c6758-8675-43f8-8da0-80f32a490537\") " pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019305 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-hostroot\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:06.019330 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:31:06.519308609 +0000 UTC m=+2.006704023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019351 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47e464a7-c6bb-45fa-89b9-a4c5eb701478-host\") pod \"node-ca-gjvt4\" (UID: \"47e464a7-c6bb-45fa-89b9-a4c5eb701478\") " pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019385 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-hostroot\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019441 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-os-release\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019468 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019474 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25b068f3-c4ca-4d18-b74c-1b8d653d27c8-tmp-dir\") pod \"node-resolver-l2nsz\" (UID: \"25b068f3-c4ca-4d18-b74c-1b8d653d27c8\") " pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019502 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-sys\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019517 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-sysctl-conf\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019542 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-run-k8s-cni-cncf-io\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019559 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b035aac5-e945-44dd-a4f6-131fca47ad48-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019600 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-run-k8s-cni-cncf-io\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019563 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019617 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-sysctl-conf\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.020172 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019628 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-lib-modules\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019663 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-system-cni-dir\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019683 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d20d9255-7e0d-4fd4-a553-18bb3769366c-cni-binary-copy\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019681 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-system-cni-dir\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019703 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-etc-kubernetes\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019738 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019775 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcq2g\" (UniqueName: \"kubernetes.io/projected/f4f1ee0d-0554-4134-b502-08c8aefca7ec-kube-api-access-lcq2g\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019799 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-cnibin\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019845 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-run-netns\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019876 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-var-lib-kubelet\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019906 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-cnibin\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019926 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-registration-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019954 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6bcf0af6-b67a-4338-b2af-6431464934f1-konnectivity-ca\") pod \"konnectivity-agent-4z64k\" (UID: \"6bcf0af6-b67a-4338-b2af-6431464934f1\") " pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019964 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-var-lib-kubelet\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019799 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-etc-kubernetes\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019918 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-run-netns\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019992 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-var-lib-kubelet\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.019875 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.020988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020022 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qlrs\" (UniqueName: \"kubernetes.io/projected/6e60d724-3791-4386-84d2-f58f064342ef-kube-api-access-5qlrs\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020041 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-registration-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020082 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-var-lib-kubelet\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020115 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-os-release\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020147 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-var-lib-cni-multus\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020177 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-socket-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020203 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b035aac5-e945-44dd-a4f6-131fca47ad48-cni-binary-copy\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020233 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-var-lib-cni-multus\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020234 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d20d9255-7e0d-4fd4-a553-18bb3769366c-cni-binary-copy\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020266 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-cni-dir\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020333 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0a254df8-e862-4e27-bb25-3139b2de01d0-socket-dir\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020336 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-var-lib-cni-bin\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020380 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhd9l\" (UniqueName: \"kubernetes.io/projected/524c6758-8675-43f8-8da0-80f32a490537-kube-api-access-jhd9l\") pod \"iptables-alerter-frwtg\" (UID: \"524c6758-8675-43f8-8da0-80f32a490537\") " pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020381 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-var-lib-cni-bin\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020411 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-sysconfig\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020446 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-run\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020463 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-sysconfig\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.021849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020474 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-modprobe-d\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020381 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-cni-dir\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020515 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-run\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020523 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6bcf0af6-b67a-4338-b2af-6431464934f1-konnectivity-ca\") pod \"konnectivity-agent-4z64k\" (UID: \"6bcf0af6-b67a-4338-b2af-6431464934f1\") " pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020544 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-host\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020567 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e60d724-3791-4386-84d2-f58f064342ef-tmp\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020571 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-modprobe-d\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020585 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcr5p\" (UniqueName: \"kubernetes.io/projected/25b068f3-c4ca-4d18-b74c-1b8d653d27c8-kube-api-access-tcr5p\") pod \"node-resolver-l2nsz\" (UID: \"25b068f3-c4ca-4d18-b74c-1b8d653d27c8\") " pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020604 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b035aac5-e945-44dd-a4f6-131fca47ad48-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020621 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-host\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020636 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-run-multus-certs\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020668 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-sysctl-d\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020714 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b035aac5-e945-44dd-a4f6-131fca47ad48-cni-binary-copy\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020710 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/47e464a7-c6bb-45fa-89b9-a4c5eb701478-serviceca\") pod \"node-ca-gjvt4\" (UID: \"47e464a7-c6bb-45fa-89b9-a4c5eb701478\") " pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020773 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-system-cni-dir\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020782 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-host-run-multus-certs\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020834 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/524c6758-8675-43f8-8da0-80f32a490537-host-slash\") pod \"iptables-alerter-frwtg\" (UID: \"524c6758-8675-43f8-8da0-80f32a490537\") " pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020864 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-daemon-config\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.022650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020880 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d20d9255-7e0d-4fd4-a553-18bb3769366c-system-cni-dir\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.023401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.020900 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6e60d724-3791-4386-84d2-f58f064342ef-etc-sysctl-d\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.023401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.021071 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/524c6758-8675-43f8-8da0-80f32a490537-host-slash\") pod \"iptables-alerter-frwtg\" (UID: \"524c6758-8675-43f8-8da0-80f32a490537\") " pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:06.023401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.021129 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b035aac5-e945-44dd-a4f6-131fca47ad48-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.023401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.021029 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b035aac5-e945-44dd-a4f6-131fca47ad48-os-release\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.023401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.021277 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/47e464a7-c6bb-45fa-89b9-a4c5eb701478-serviceca\") pod \"node-ca-gjvt4\" (UID: \"47e464a7-c6bb-45fa-89b9-a4c5eb701478\") " pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:06.023401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.021513 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6e60d724-3791-4386-84d2-f58f064342ef-etc-tuned\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.023401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.021520 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d20d9255-7e0d-4fd4-a553-18bb3769366c-multus-daemon-config\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.023401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.021591 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6bcf0af6-b67a-4338-b2af-6431464934f1-agent-certs\") pod \"konnectivity-agent-4z64k\" (UID: \"6bcf0af6-b67a-4338-b2af-6431464934f1\") " pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:06.023401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.023291 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e60d724-3791-4386-84d2-f58f064342ef-tmp\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.031357 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.031331 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk9bs\" (UniqueName: \"kubernetes.io/projected/0a254df8-e862-4e27-bb25-3139b2de01d0-kube-api-access-zk9bs\") pod \"aws-ebs-csi-driver-node-tl7c7\" (UID: \"0a254df8-e862-4e27-bb25-3139b2de01d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.032473 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:06.032179 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:31:06.032473 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:06.032217 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:31:06.032473 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:06.032230 2547 projected.go:196] Error preparing data for projected volume kube-api-access-wgj9v for pod openshift-network-diagnostics/network-check-target-pqqpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:06.032473 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:06.032312 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v podName:d4147e92-e8f2-4a19-8fe7-fdd1c8a52174 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:06.532288804 +0000 UTC m=+2.019684219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wgj9v" (UniqueName: "kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v") pod "network-check-target-pqqpn" (UID: "d4147e92-e8f2-4a19-8fe7-fdd1c8a52174") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:06.034546 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.034491 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhd9l\" (UniqueName: \"kubernetes.io/projected/524c6758-8675-43f8-8da0-80f32a490537-kube-api-access-jhd9l\") pod \"iptables-alerter-frwtg\" (UID: \"524c6758-8675-43f8-8da0-80f32a490537\") " pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:06.034647 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.034628 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcr5p\" (UniqueName: \"kubernetes.io/projected/25b068f3-c4ca-4d18-b74c-1b8d653d27c8-kube-api-access-tcr5p\") pod \"node-resolver-l2nsz\" (UID: \"25b068f3-c4ca-4d18-b74c-1b8d653d27c8\") " pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:06.034762 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.034744 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qlrs\" (UniqueName: \"kubernetes.io/projected/6e60d724-3791-4386-84d2-f58f064342ef-kube-api-access-5qlrs\") pod \"tuned-rj46c\" (UID: \"6e60d724-3791-4386-84d2-f58f064342ef\") " pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.034958 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.034942 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnt6d\" (UniqueName: \"kubernetes.io/projected/b035aac5-e945-44dd-a4f6-131fca47ad48-kube-api-access-lnt6d\") pod \"multus-additional-cni-plugins-bswhr\" (UID: \"b035aac5-e945-44dd-a4f6-131fca47ad48\") " pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.035002 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.034963 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndgdf\" (UniqueName: \"kubernetes.io/projected/d20d9255-7e0d-4fd4-a553-18bb3769366c-kube-api-access-ndgdf\") pod \"multus-szhvf\" (UID: \"d20d9255-7e0d-4fd4-a553-18bb3769366c\") " pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.035037 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.034965 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcq2g\" (UniqueName: \"kubernetes.io/projected/f4f1ee0d-0554-4134-b502-08c8aefca7ec-kube-api-access-lcq2g\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:06.035430 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.035415 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k8sn\" (UniqueName: \"kubernetes.io/projected/47e464a7-c6bb-45fa-89b9-a4c5eb701478-kube-api-access-6k8sn\") pod \"node-ca-gjvt4\" (UID: \"47e464a7-c6bb-45fa-89b9-a4c5eb701478\") " pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:06.132315 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.132226 2547 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:31:06.135612 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.135589 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:06.145412 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:06.145388 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818e1edd_f50a_43b0_86a5_03c2c318e468.slice/crio-47d8996bf752f836f08b075c3e8ea6853e04004d6fac581cdae23d3a255bb4de WatchSource:0}: Error finding container 47d8996bf752f836f08b075c3e8ea6853e04004d6fac581cdae23d3a255bb4de: Status 404 returned error can't find the container with id 47d8996bf752f836f08b075c3e8ea6853e04004d6fac581cdae23d3a255bb4de Jul 02 21:31:06.151667 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.151648 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" Jul 02 21:31:06.160310 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:06.160287 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a254df8_e862_4e27_bb25_3139b2de01d0.slice/crio-f412a1108b937b6fad89a494ee9f63ed8e9de44248d19d7a22bccc1c78d75459 WatchSource:0}: Error finding container f412a1108b937b6fad89a494ee9f63ed8e9de44248d19d7a22bccc1c78d75459: Status 404 returned error can't find the container with id f412a1108b937b6fad89a494ee9f63ed8e9de44248d19d7a22bccc1c78d75459 Jul 02 21:31:06.166337 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.166319 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rj46c" Jul 02 21:31:06.173990 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:06.173968 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e60d724_3791_4386_84d2_f58f064342ef.slice/crio-761d66320fa85ee9a917827e54828b51f9061f470437d627e4e7920eaddd4cf4 WatchSource:0}: Error finding container 761d66320fa85ee9a917827e54828b51f9061f470437d627e4e7920eaddd4cf4: Status 404 returned error can't find the container with id 761d66320fa85ee9a917827e54828b51f9061f470437d627e4e7920eaddd4cf4 Jul 02 21:31:06.180340 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.180324 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l2nsz" Jul 02 21:31:06.187707 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:06.187684 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b068f3_c4ca_4d18_b74c_1b8d653d27c8.slice/crio-75f12e866190665540ba96f543dea46ad6c65a47a0b6dc0ea81483051ecbb9c3 WatchSource:0}: Error finding container 75f12e866190665540ba96f543dea46ad6c65a47a0b6dc0ea81483051ecbb9c3: Status 404 returned error can't find the container with id 75f12e866190665540ba96f543dea46ad6c65a47a0b6dc0ea81483051ecbb9c3 Jul 02 21:31:06.195948 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.195932 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gjvt4" Jul 02 21:31:06.203618 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:06.203598 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e464a7_c6bb_45fa_89b9_a4c5eb701478.slice/crio-92b868a51d423d98e29bb4ae6900503f7a0cdbe30c55fb273da8cda97f769f34 WatchSource:0}: Error finding container 92b868a51d423d98e29bb4ae6900503f7a0cdbe30c55fb273da8cda97f769f34: Status 404 returned error can't find the container with id 92b868a51d423d98e29bb4ae6900503f7a0cdbe30c55fb273da8cda97f769f34 Jul 02 21:31:06.212501 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.212481 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bswhr" Jul 02 21:31:06.220753 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:06.220730 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb035aac5_e945_44dd_a4f6_131fca47ad48.slice/crio-a74ce839d543dbbb20a31278f7b686bd54b35b5b026b6662e4f54091b5d05072 WatchSource:0}: Error finding container a74ce839d543dbbb20a31278f7b686bd54b35b5b026b6662e4f54091b5d05072: Status 404 returned error can't find the container with id a74ce839d543dbbb20a31278f7b686bd54b35b5b026b6662e4f54091b5d05072 Jul 02 21:31:06.222716 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.222693 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:06.230957 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:06.230933 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bcf0af6_b67a_4338_b2af_6431464934f1.slice/crio-c480896acadb9d6cc24e8879700dcfebe5caf162e9240c398f5aa4bce39566c4 WatchSource:0}: Error finding container c480896acadb9d6cc24e8879700dcfebe5caf162e9240c398f5aa4bce39566c4: Status 404 returned error can't find the container with id c480896acadb9d6cc24e8879700dcfebe5caf162e9240c398f5aa4bce39566c4 Jul 02 21:31:06.243104 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.243078 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-szhvf" Jul 02 21:31:06.248921 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.248898 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-frwtg" Jul 02 21:31:06.253783 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:06.253759 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd20d9255_7e0d_4fd4_a553_18bb3769366c.slice/crio-b8f7ad1a1d8c37d9e5d7cf705b3839f5ce3260e11f11170a53e0d1450ed07c73 WatchSource:0}: Error finding container b8f7ad1a1d8c37d9e5d7cf705b3839f5ce3260e11f11170a53e0d1450ed07c73: Status 404 returned error can't find the container with id b8f7ad1a1d8c37d9e5d7cf705b3839f5ce3260e11f11170a53e0d1450ed07c73 Jul 02 21:31:06.257481 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:06.257467 2547 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:31:06.258192 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:06.258173 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod524c6758_8675_43f8_8da0_80f32a490537.slice/crio-96b47b0ba4eabab920f4241d4d5902b78eb6661dde24e8241a0a71637f889859 WatchSource:0}: Error finding container 96b47b0ba4eabab920f4241d4d5902b78eb6661dde24e8241a0a71637f889859: Status 404 returned error can't find the container with id 96b47b0ba4eabab920f4241d4d5902b78eb6661dde24e8241a0a71637f889859 Jul 02 21:31:07.460708 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:07.460628 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:07.460852 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:07.460752 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:07.460852 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:07.460821 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret podName:e6c4320e-5777-4358-8dac-d43f1f4f411e nodeName:}" failed. No retries permitted until 2026-07-02 21:31:08.460800006 +0000 UTC m=+2.910638627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret") pod "global-pull-secret-syncer-hsd7b" (UID: "e6c4320e-5777-4358-8dac-d43f1f4f411e") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:07.561257 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:07.561152 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:07.561257 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:07.561253 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:07.561489 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:07.561313 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:31:08.561296214 +0000 UTC m=+3.011134819 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:07.662197 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:07.662163 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:07.662399 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:07.662363 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:31:07.662399 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:07.662398 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:31:07.662513 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:07.662413 2547 projected.go:196] Error preparing data for projected volume kube-api-access-wgj9v for pod openshift-network-diagnostics/network-check-target-pqqpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:07.662513 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:07.662472 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v podName:d4147e92-e8f2-4a19-8fe7-fdd1c8a52174 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:08.662453406 +0000 UTC m=+3.112292029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wgj9v" (UniqueName: "kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v") pod "network-check-target-pqqpn" (UID: "d4147e92-e8f2-4a19-8fe7-fdd1c8a52174") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:07.898327 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:07.898079 2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 21:26:06 +0000 UTC" deadline="2027-12-19 16:52:55.449070235 +0000 UTC" Jul 02 21:31:07.898327 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:07.898260 2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12835h21m47.550815976s" Jul 02 21:31:07.971002 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:07.970658 2547 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:31:08.033015 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.032896 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-frwtg" event={"ID":"524c6758-8675-43f8-8da0-80f32a490537","Type":"ContainerStarted","Data":"96b47b0ba4eabab920f4241d4d5902b78eb6661dde24e8241a0a71637f889859"} Jul 02 21:31:08.055420 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.055370 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-szhvf" event={"ID":"d20d9255-7e0d-4fd4-a553-18bb3769366c","Type":"ContainerStarted","Data":"b8f7ad1a1d8c37d9e5d7cf705b3839f5ce3260e11f11170a53e0d1450ed07c73"} Jul 02 21:31:08.081050 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.080949 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gjvt4" event={"ID":"47e464a7-c6bb-45fa-89b9-a4c5eb701478","Type":"ContainerStarted","Data":"92b868a51d423d98e29bb4ae6900503f7a0cdbe30c55fb273da8cda97f769f34"} Jul 02 21:31:08.114358 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.114321 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l2nsz" event={"ID":"25b068f3-c4ca-4d18-b74c-1b8d653d27c8","Type":"ContainerStarted","Data":"75f12e866190665540ba96f543dea46ad6c65a47a0b6dc0ea81483051ecbb9c3"} Jul 02 21:31:08.138067 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.137984 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rj46c" event={"ID":"6e60d724-3791-4386-84d2-f58f064342ef","Type":"ContainerStarted","Data":"761d66320fa85ee9a917827e54828b51f9061f470437d627e4e7920eaddd4cf4"} Jul 02 21:31:08.152087 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.151928 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" event={"ID":"0a254df8-e862-4e27-bb25-3139b2de01d0","Type":"ContainerStarted","Data":"f412a1108b937b6fad89a494ee9f63ed8e9de44248d19d7a22bccc1c78d75459"} Jul 02 21:31:08.182047 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.181965 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerStarted","Data":"47d8996bf752f836f08b075c3e8ea6853e04004d6fac581cdae23d3a255bb4de"} Jul 02 21:31:08.204885 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.204844 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4z64k" event={"ID":"6bcf0af6-b67a-4338-b2af-6431464934f1","Type":"ContainerStarted","Data":"c480896acadb9d6cc24e8879700dcfebe5caf162e9240c398f5aa4bce39566c4"} Jul 02 21:31:08.237210 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.237171 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bswhr" event={"ID":"b035aac5-e945-44dd-a4f6-131fca47ad48","Type":"ContainerStarted","Data":"a74ce839d543dbbb20a31278f7b686bd54b35b5b026b6662e4f54091b5d05072"} Jul 02 21:31:08.265399 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.265339 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal" event={"ID":"c48cb01bf393d73b784190e0bda418c3","Type":"ContainerStarted","Data":"d0ff255fe6122bbe3102e1f7790d7d4b71e9fd6ebff317bab7637a100b358e07"} Jul 02 21:31:08.296459 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.296417 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" event={"ID":"47de5f2930588a3e744010cfb8dff070","Type":"ContainerStarted","Data":"ca2b9c85afaf5aa020f20f556d6930fc462ea58a8aa6de00ee6e9075f49d23a3"} Jul 02 21:31:08.476792 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.476707 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:08.476963 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:08.476927 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:08.477022 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:08.476990 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret podName:e6c4320e-5777-4358-8dac-d43f1f4f411e nodeName:}" failed. No retries permitted until 2026-07-02 21:31:10.476972305 +0000 UTC m=+4.926810910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret") pod "global-pull-secret-syncer-hsd7b" (UID: "e6c4320e-5777-4358-8dac-d43f1f4f411e") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:08.577906 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.577870 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:08.578074 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:08.578047 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:08.578136 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:08.578108 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:31:10.578090445 +0000 UTC m=+5.027929062 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:08.679369 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.678715 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:08.679369 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:08.678886 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:31:08.679369 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:08.678910 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:31:08.679369 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:08.678922 2547 projected.go:196] Error preparing data for projected volume kube-api-access-wgj9v for pod openshift-network-diagnostics/network-check-target-pqqpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:08.679369 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:08.678978 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v podName:d4147e92-e8f2-4a19-8fe7-fdd1c8a52174 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:10.678958392 +0000 UTC m=+5.128797012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wgj9v" (UniqueName: "kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v") pod "network-check-target-pqqpn" (UID: "d4147e92-e8f2-4a19-8fe7-fdd1c8a52174") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:08.898996 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.898925 2547 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 21:26:06 +0000 UTC" deadline="2028-04-13 10:49:46.644634576 +0000 UTC" Jul 02 21:31:08.898996 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:08.898963 2547 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15613h18m37.745677379s" Jul 02 21:31:09.003139 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:09.002919 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:09.003957 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:09.003590 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:09.003957 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:09.003637 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:09.003957 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:09.003730 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:09.003957 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:09.003776 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:09.003957 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:09.003922 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:09.117852 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:09.117632 2547 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 21:31:10.495072 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:10.495038 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:10.495597 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:10.495182 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:10.495597 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:10.495242 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret podName:e6c4320e-5777-4358-8dac-d43f1f4f411e nodeName:}" failed. No retries permitted until 2026-07-02 21:31:14.495224198 +0000 UTC m=+8.945062814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret") pod "global-pull-secret-syncer-hsd7b" (UID: "e6c4320e-5777-4358-8dac-d43f1f4f411e") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:10.595778 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:10.595746 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:10.595964 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:10.595947 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:10.596027 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:10.596012 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:31:14.595991728 +0000 UTC m=+9.045830344 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:10.696185 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:10.696094 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:10.696355 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:10.696284 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:31:10.696355 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:10.696308 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:31:10.696355 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:10.696321 2547 projected.go:196] Error preparing data for projected volume kube-api-access-wgj9v for pod openshift-network-diagnostics/network-check-target-pqqpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:10.696493 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:10.696374 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v podName:d4147e92-e8f2-4a19-8fe7-fdd1c8a52174 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:14.696356631 +0000 UTC m=+9.146195245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wgj9v" (UniqueName: "kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v") pod "network-check-target-pqqpn" (UID: "d4147e92-e8f2-4a19-8fe7-fdd1c8a52174") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:11.002098 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:11.002068 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:11.002301 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:11.002073 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:11.002365 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:11.002320 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:11.002365 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:11.002077 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:11.002499 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:11.002205 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:11.002499 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:11.002457 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:13.002351 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:13.002309 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:13.002830 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:13.002357 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:13.002830 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:13.002364 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:13.002830 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:13.002528 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:13.002830 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:13.002676 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:13.002830 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:13.002755 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:14.534198 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:14.534163 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:14.534603 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:14.534345 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:14.534603 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:14.534437 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret podName:e6c4320e-5777-4358-8dac-d43f1f4f411e nodeName:}" failed. No retries permitted until 2026-07-02 21:31:22.534417025 +0000 UTC m=+16.984255628 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret") pod "global-pull-secret-syncer-hsd7b" (UID: "e6c4320e-5777-4358-8dac-d43f1f4f411e") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:14.634808 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:14.634771 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:14.635001 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:14.634916 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:14.635001 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:14.634985 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:31:22.634965455 +0000 UTC m=+17.084804075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:14.736080 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:14.735980 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:14.736282 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:14.736181 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:31:14.736282 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:14.736205 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:31:14.736282 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:14.736218 2547 projected.go:196] Error preparing data for projected volume kube-api-access-wgj9v for pod openshift-network-diagnostics/network-check-target-pqqpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:14.736484 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:14.736288 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v podName:d4147e92-e8f2-4a19-8fe7-fdd1c8a52174 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:22.736266797 +0000 UTC m=+17.186105422 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wgj9v" (UniqueName: "kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v") pod "network-check-target-pqqpn" (UID: "d4147e92-e8f2-4a19-8fe7-fdd1c8a52174") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:15.002976 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:15.002706 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:15.002976 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:15.002720 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:15.002976 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:15.002849 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:15.002976 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:15.002927 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:15.002976 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:15.002967 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:15.003355 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:15.003158 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:17.003043 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:17.003001 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:17.003545 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:17.003002 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:17.003545 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:17.003159 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:17.003545 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:17.003011 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:17.003545 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:17.003233 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:17.003545 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:17.003348 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:19.002123 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:19.002039 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:19.002123 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:19.002085 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:19.002603 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:19.002040 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:19.002603 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:19.002170 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:19.002603 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:19.002243 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:19.002603 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:19.002331 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:21.002744 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:21.002541 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:21.003207 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:21.002550 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:21.003207 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:21.002838 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:21.003207 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:21.002600 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:21.003207 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:21.002933 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:21.003207 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:21.003004 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:22.594526 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:22.594495 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:22.594921 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:22.594616 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:22.594921 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:22.594686 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret podName:e6c4320e-5777-4358-8dac-d43f1f4f411e nodeName:}" failed. No retries permitted until 2026-07-02 21:31:38.594664424 +0000 UTC m=+33.044503026 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret") pod "global-pull-secret-syncer-hsd7b" (UID: "e6c4320e-5777-4358-8dac-d43f1f4f411e") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:22.695007 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:22.694969 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:22.695194 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:22.695176 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:22.695267 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:22.695255 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:31:38.695233662 +0000 UTC m=+33.145072279 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:22.795546 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:22.795508 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:22.795740 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:22.795720 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:31:22.795797 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:22.795749 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:31:22.795797 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:22.795763 2547 projected.go:196] Error preparing data for projected volume kube-api-access-wgj9v for pod openshift-network-diagnostics/network-check-target-pqqpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:22.795895 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:22.795837 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v podName:d4147e92-e8f2-4a19-8fe7-fdd1c8a52174 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:38.795817374 +0000 UTC m=+33.245655985 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wgj9v" (UniqueName: "kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v") pod "network-check-target-pqqpn" (UID: "d4147e92-e8f2-4a19-8fe7-fdd1c8a52174") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:23.002667 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:23.002577 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:23.002667 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:23.002613 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:23.002667 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:23.002613 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:23.002965 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:23.002723 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:23.002965 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:23.002776 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:23.002965 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:23.002848 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:25.002561 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:25.002524 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:25.003014 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:25.002529 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:25.003014 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:25.002636 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:25.003014 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:25.002529 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:25.003014 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:25.002713 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:25.003014 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:25.002782 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:27.002573 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:27.002534 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:27.002970 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:27.002627 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:27.002970 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:27.002640 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:27.002970 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:27.002547 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:27.002970 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:27.002751 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:27.002970 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:27.002853 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:28.338711 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.338522 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" event={"ID":"0a254df8-e862-4e27-bb25-3139b2de01d0","Type":"ContainerStarted","Data":"ac36efd28ede72f03d00877197304527f508665a8c2ef8bdc182025853f79491"} Jul 02 21:31:28.341116 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.341096 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:31:28.341452 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.341431 2547 generic.go:356] "Generic (PLEG): container finished" podID="818e1edd-f50a-43b0-86a5-03c2c318e468" containerID="0c6d7929a0d7e1962d70fabc70318aeef13904ec0d00cddfb5e59129feaf055d" exitCode=1 Jul 02 21:31:28.341527 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.341506 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerStarted","Data":"57172ed6c52e046e999502854e9b28bc1f4a2627c7fce843a8643a233d0d4ffe"} Jul 02 21:31:28.341587 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.341573 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerStarted","Data":"4bd5802ccfde2b0791e898eae6419490a80dfbd284b5695144ec30c62404e75f"} Jul 02 21:31:28.341628 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.341613 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerStarted","Data":"c88af6f9c66d4328a8b1baa34f922f0a6c1f741f7c9aa83ba661270e06255973"} Jul 02 21:31:28.341666 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.341649 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerStarted","Data":"b9e497164015840e49a7e9f14265789737a31a945cf738e363436dfb8a451c5e"} Jul 02 21:31:28.341714 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.341692 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerDied","Data":"0c6d7929a0d7e1962d70fabc70318aeef13904ec0d00cddfb5e59129feaf055d"} Jul 02 21:31:28.341766 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.341726 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerStarted","Data":"704c7952bf8ef714518a0fc4f227a31597c8d83096361a941444bde39d24116e"} Jul 02 21:31:28.342851 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.342823 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4z64k" event={"ID":"6bcf0af6-b67a-4338-b2af-6431464934f1","Type":"ContainerStarted","Data":"6f7c962f47f2bcee5604e4a7695f96a4c11ab18352652e7365b9e9ed37509688"} Jul 02 21:31:28.344295 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.344273 2547 generic.go:356] "Generic (PLEG): container finished" podID="b035aac5-e945-44dd-a4f6-131fca47ad48" containerID="3e6737fcfdd8da1af812f0ccc27dc3d2a300e7e4294457f55af6a97ebb8d669f" exitCode=0 Jul 02 21:31:28.344414 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.344338 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bswhr" event={"ID":"b035aac5-e945-44dd-a4f6-131fca47ad48","Type":"ContainerDied","Data":"3e6737fcfdd8da1af812f0ccc27dc3d2a300e7e4294457f55af6a97ebb8d669f"} Jul 02 21:31:28.345811 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.345787 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal" event={"ID":"c48cb01bf393d73b784190e0bda418c3","Type":"ContainerStarted","Data":"a75ad12fc035c31f07298bcb93d85b66aa4963b9868d32bf035f694b577e5d15"} Jul 02 21:31:28.347352 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.347333 2547 generic.go:356] "Generic (PLEG): container finished" podID="47de5f2930588a3e744010cfb8dff070" containerID="24fc0b5179f263553fc3402f28250b4a6f4201c653009088be420cbaec75ecbf" exitCode=0 Jul 02 21:31:28.347456 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.347362 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" event={"ID":"47de5f2930588a3e744010cfb8dff070","Type":"ContainerDied","Data":"24fc0b5179f263553fc3402f28250b4a6f4201c653009088be420cbaec75ecbf"} Jul 02 21:31:28.348954 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.348931 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-szhvf" event={"ID":"d20d9255-7e0d-4fd4-a553-18bb3769366c","Type":"ContainerStarted","Data":"2dbbd0f4f8c970feffb8cc54530d8d365c9e721154bbf798e15553b76e90d0cb"} Jul 02 21:31:28.350351 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.350331 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gjvt4" event={"ID":"47e464a7-c6bb-45fa-89b9-a4c5eb701478","Type":"ContainerStarted","Data":"5cda83030cf74b55cf7de44d4de797d111caa4f6340729b99dea439fb93fa3e8"} Jul 02 21:31:28.351626 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.351605 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l2nsz" event={"ID":"25b068f3-c4ca-4d18-b74c-1b8d653d27c8","Type":"ContainerStarted","Data":"2af2dcb66ddd14c6117482b32ba3586d1c7664162c530cfed9c9dfb2ebc7f2ff"} Jul 02 21:31:28.353055 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.353038 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rj46c" event={"ID":"6e60d724-3791-4386-84d2-f58f064342ef","Type":"ContainerStarted","Data":"2bc698ee03b9c22015a7ef3d7d0ca9216ed4de83783888a496541af6d1bd329e"} Jul 02 21:31:28.361713 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.361675 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4z64k" podStartSLOduration=2.527115603 podStartE2EDuration="22.361661765s" totalImagesPullingTime="19.834546162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:28.361166044 +0000 UTC m=+22.811004669" watchObservedRunningTime="2026-07-02 21:31:28.361661765 +0000 UTC m=+22.811500388" Jul 02 21:31:28.399808 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.399545 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l2nsz" podStartSLOduration=2.521787652 podStartE2EDuration="22.399524919s" totalImagesPullingTime="19.877737267s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:28.398806519 +0000 UTC m=+22.848645144" watchObservedRunningTime="2026-07-02 21:31:28.399524919 +0000 UTC m=+22.849363543" Jul 02 21:31:28.418961 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.418749 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-szhvf" podStartSLOduration=2.29080337 podStartE2EDuration="22.41872983s" totalImagesPullingTime="20.12792646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:28.417982049 +0000 UTC m=+22.867820673" watchObservedRunningTime="2026-07-02 21:31:28.41872983 +0000 UTC m=+22.868568455" Jul 02 21:31:28.433176 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.433124 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-35.ec2.internal" podStartSLOduration=22.433109401 podStartE2EDuration="22.433109401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:28.432997201 +0000 UTC m=+22.882835824" watchObservedRunningTime="2026-07-02 21:31:28.433109401 +0000 UTC m=+22.882948024" Jul 02 21:31:28.451962 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.451918 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rj46c" podStartSLOduration=2.5584007570000002 podStartE2EDuration="22.451902851s" totalImagesPullingTime="19.893502094s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:28.451624194 +0000 UTC m=+22.901462819" watchObservedRunningTime="2026-07-02 21:31:28.451902851 +0000 UTC m=+22.901741474" Jul 02 21:31:28.500136 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:28.500089 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gjvt4" podStartSLOduration=2.638589422 podStartE2EDuration="22.500071697s" totalImagesPullingTime="19.861482275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:28.467063653 +0000 UTC m=+22.916902287" watchObservedRunningTime="2026-07-02 21:31:28.500071697 +0000 UTC m=+22.949910315" Jul 02 21:31:29.002768 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.002732 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:29.002971 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.002782 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:29.002971 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.002738 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:29.002971 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:29.002878 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:29.003138 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:29.002981 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:29.003138 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:29.003083 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:29.357766 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.357720 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" event={"ID":"47de5f2930588a3e744010cfb8dff070","Type":"ContainerStarted","Data":"897f01959ca63ec03efff1e3a66080ede99cf7e6c933599e053f3726488893fa"} Jul 02 21:31:29.359698 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.359637 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-frwtg" event={"ID":"524c6758-8675-43f8-8da0-80f32a490537","Type":"ContainerStarted","Data":"6a2c44c226d566e22627e747daf90be42d80c72e5912ce8291c5128d083a8327"} Jul 02 21:31:29.375045 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.374995 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-35.ec2.internal" podStartSLOduration=23.374979318 podStartE2EDuration="23.374979318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:29.374404414 +0000 UTC m=+23.824243031" watchObservedRunningTime="2026-07-02 21:31:29.374979318 +0000 UTC m=+23.824817941" Jul 02 21:31:29.389499 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.389441 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-frwtg" podStartSLOduration=3.5816264909999997 podStartE2EDuration="23.389420413s" totalImagesPullingTime="19.807793922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:29.389404845 +0000 UTC m=+23.839243472" watchObservedRunningTime="2026-07-02 21:31:29.389420413 +0000 UTC m=+23.839259038" Jul 02 21:31:29.500161 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.500119 2547 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 21:31:29.914159 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.914055 2547 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T21:31:29.500142909Z","UUID":"0847c9df-2f3b-4779-b2f5-c4f914c40117","Handler":null,"Name":"","Endpoint":""} Jul 02 21:31:29.915943 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.915919 2547 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 21:31:29.916138 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:29.915950 2547 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 21:31:30.031926 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:30.031885 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:30.032561 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:30.032539 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:30.364803 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:30.364751 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" event={"ID":"0a254df8-e862-4e27-bb25-3139b2de01d0","Type":"ContainerStarted","Data":"559a5d02a7a80e8b6b4b3e978e9edb54034c77b178c688160a3820bddb08e8d0"} Jul 02 21:31:30.368965 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:30.368937 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:31:30.369483 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:30.369453 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerStarted","Data":"999b21ae233403125e5bfd74a564c0eedada23b4dd08d807241dc6554b8c2493"} Jul 02 21:31:30.369854 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:30.369830 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:30.370425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:30.370409 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4z64k" Jul 02 21:31:31.002811 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:31.002730 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:31.002999 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:31.002729 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:31.002999 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:31.002859 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:31.002999 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:31.002739 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:31.002999 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:31.002982 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:31.003173 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:31.003028 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:31.374579 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:31.374546 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" event={"ID":"0a254df8-e862-4e27-bb25-3139b2de01d0","Type":"ContainerStarted","Data":"f729b5391cf50c2e7dd9dd927f7d66b3d71a7e22b603e056fe639fb490dde438"} Jul 02 21:31:31.397696 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:31.397647 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tl7c7" podStartSLOduration=3.156723907 podStartE2EDuration="25.397630318s" totalImagesPullingTime="22.240906411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:31.397504897 +0000 UTC m=+25.847343523" watchObservedRunningTime="2026-07-02 21:31:31.397630318 +0000 UTC m=+25.847468941" Jul 02 21:31:33.003171 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.002929 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:33.003816 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.002956 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:33.003816 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:33.003209 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:33.003816 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.003010 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:33.003816 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:33.003283 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:33.003816 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:33.003404 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:33.382043 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.382017 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:31:33.382394 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.382361 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerStarted","Data":"fa0e8a8d2523f5cdf42a4140c521525dd0e8cf930fd7334e507f03326308a550"} Jul 02 21:31:33.382681 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.382662 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:33.382736 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.382728 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:33.382870 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.382848 2547 scope.go:117] "RemoveContainer" containerID="0c6d7929a0d7e1962d70fabc70318aeef13904ec0d00cddfb5e59129feaf055d" Jul 02 21:31:33.384834 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.384799 2547 generic.go:356] "Generic (PLEG): container finished" podID="b035aac5-e945-44dd-a4f6-131fca47ad48" containerID="e104ac5ce280bef0ba1a5419109474d48e643b3aef4416e9f6d102f85165fdd4" exitCode=0 Jul 02 21:31:33.385614 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.385569 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bswhr" event={"ID":"b035aac5-e945-44dd-a4f6-131fca47ad48","Type":"ContainerDied","Data":"e104ac5ce280bef0ba1a5419109474d48e643b3aef4416e9f6d102f85165fdd4"} Jul 02 21:31:33.399581 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:33.399559 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:34.339126 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.338910 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hsd7b"] Jul 02 21:31:34.339560 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.339207 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:34.339560 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:34.339291 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:34.341796 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.341777 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-95lhp"] Jul 02 21:31:34.341898 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.341874 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:34.341999 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:34.341981 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:34.344562 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.344531 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pqqpn"] Jul 02 21:31:34.344661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.344611 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:34.344731 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:34.344703 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:34.391589 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.391570 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:31:34.391968 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.391934 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" event={"ID":"818e1edd-f50a-43b0-86a5-03c2c318e468","Type":"ContainerStarted","Data":"f3e6f95b62b23c7f7018cbeeaae9dbca7b337acf217d83ed288ddca0138881e2"} Jul 02 21:31:34.392196 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.392168 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:34.394263 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.394240 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bswhr" event={"ID":"b035aac5-e945-44dd-a4f6-131fca47ad48","Type":"ContainerStarted","Data":"b8b2db28f5827d175c321565069b460ed1af3b1aafb295e23d4620fa87871166"} Jul 02 21:31:34.406928 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.406910 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:31:34.441599 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:34.441492 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" podStartSLOduration=8.475126314 podStartE2EDuration="28.441471827s" totalImagesPullingTime="19.966345513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:34.418452782 +0000 UTC m=+28.868291407" watchObservedRunningTime="2026-07-02 21:31:34.441471827 +0000 UTC m=+28.891310452" Jul 02 21:31:35.398200 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:35.398166 2547 generic.go:356] "Generic (PLEG): container finished" podID="b035aac5-e945-44dd-a4f6-131fca47ad48" containerID="b8b2db28f5827d175c321565069b460ed1af3b1aafb295e23d4620fa87871166" exitCode=0 Jul 02 21:31:35.398771 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:35.398257 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bswhr" event={"ID":"b035aac5-e945-44dd-a4f6-131fca47ad48","Type":"ContainerDied","Data":"b8b2db28f5827d175c321565069b460ed1af3b1aafb295e23d4620fa87871166"} Jul 02 21:31:36.003804 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:36.003539 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:36.003804 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:36.003645 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:36.003804 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:36.003652 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:36.003804 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:36.003785 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:36.004146 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:36.003831 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:36.004146 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:36.003875 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:37.404779 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:37.404740 2547 generic.go:356] "Generic (PLEG): container finished" podID="b035aac5-e945-44dd-a4f6-131fca47ad48" containerID="58829469408ba07da5ddf240db0179ef07470d470aa78e6df8f531fec5b1301b" exitCode=0 Jul 02 21:31:37.405330 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:37.404806 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bswhr" event={"ID":"b035aac5-e945-44dd-a4f6-131fca47ad48","Type":"ContainerDied","Data":"58829469408ba07da5ddf240db0179ef07470d470aa78e6df8f531fec5b1301b"} Jul 02 21:31:38.002534 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:38.002499 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:38.002534 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:38.002528 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:38.002763 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:38.002504 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:38.002763 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.002644 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:38.002875 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.002758 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:38.002875 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.002860 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:38.609636 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:38.609591 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:38.610093 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.609735 2547 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:38.610093 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.609795 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret podName:e6c4320e-5777-4358-8dac-d43f1f4f411e nodeName:}" failed. No retries permitted until 2026-07-02 21:32:10.609777015 +0000 UTC m=+65.059615616 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret") pod "global-pull-secret-syncer-hsd7b" (UID: "e6c4320e-5777-4358-8dac-d43f1f4f411e") : object "kube-system"/"original-pull-secret" not registered Jul 02 21:31:38.710521 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:38.710488 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:38.710679 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.710658 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:38.710741 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.710732 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:32:10.710713402 +0000 UTC m=+65.160552023 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 21:31:38.811533 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:38.811493 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:38.811709 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.811675 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 21:31:38.811709 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.811701 2547 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 21:31:38.811823 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.811715 2547 projected.go:196] Error preparing data for projected volume kube-api-access-wgj9v for pod openshift-network-diagnostics/network-check-target-pqqpn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:38.811823 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:38.811785 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v podName:d4147e92-e8f2-4a19-8fe7-fdd1c8a52174 nodeName:}" failed. No retries permitted until 2026-07-02 21:32:10.811770919 +0000 UTC m=+65.261609521 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wgj9v" (UniqueName: "kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v") pod "network-check-target-pqqpn" (UID: "d4147e92-e8f2-4a19-8fe7-fdd1c8a52174") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 21:31:40.002986 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:40.002952 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:40.003466 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:40.002951 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:40.003466 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:40.003083 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hsd7b" podUID="e6c4320e-5777-4358-8dac-d43f1f4f411e" Jul 02 21:31:40.003466 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:40.002951 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:40.003466 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:40.003184 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:31:40.003466 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:40.003249 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pqqpn" podUID="d4147e92-e8f2-4a19-8fe7-fdd1c8a52174" Jul 02 21:31:41.408492 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.408465 2547 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-35.ec2.internal" event="NodeReady" Jul 02 21:31:41.408950 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.408638 2547 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 21:31:41.444437 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.444402 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7568cffd97-p78mw"] Jul 02 21:31:41.468013 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.467985 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7568cffd97-p78mw"] Jul 02 21:31:41.468161 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.468087 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.471430 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.471199 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn"] Jul 02 21:31:41.472703 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.472681 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 21:31:41.473094 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.473080 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b4gbg\"" Jul 02 21:31:41.473947 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.473931 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 21:31:41.474021 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.473992 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 21:31:41.477702 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.477682 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 21:31:41.494207 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.494185 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wvj7q"] Jul 02 21:31:41.494337 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.494322 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:41.498077 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.497956 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-24f84\"" Jul 02 21:31:41.498077 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.498058 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 02 21:31:41.498267 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.498152 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 02 21:31:41.510181 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.510158 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-grgfz"] Jul 02 21:31:41.510278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.510257 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.512588 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.512570 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 21:31:41.512700 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.512601 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 21:31:41.512700 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.512638 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-knz6t\"" Jul 02 21:31:41.524459 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.524433 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wvj7q"] Jul 02 21:31:41.524534 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.524484 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-grgfz"] Jul 02 21:31:41.524534 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.524503 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn"] Jul 02 21:31:41.524618 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.524580 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-grgfz" Jul 02 21:31:41.527011 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.526991 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 21:31:41.527113 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.527049 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gnsm2\"" Jul 02 21:31:41.527113 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.527049 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 21:31:41.527205 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.527191 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 21:31:41.632856 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.632825 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.633010 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.632886 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbvlc\" (UniqueName: \"kubernetes.io/projected/ac732629-79f4-4eea-95f4-f5e748d23c05-kube-api-access-jbvlc\") pod \"ingress-canary-grgfz\" (UID: \"ac732629-79f4-4eea-95f4-f5e748d23c05\") " pod="openshift-ingress-canary/ingress-canary-grgfz" Jul 02 21:31:41.633010 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.632908 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/701b2907-f252-40d2-a8f9-b22923946c39-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:41.633010 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.632995 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-installation-pull-secrets\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.633143 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633059 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-image-registry-private-configuration\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.633143 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633084 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:41.633143 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633106 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-trusted-ca\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.633143 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633133 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/33485243-8457-4ab8-83a2-3ad6cc1d98f9-config-volume\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.633314 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633155 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-registry-certificates\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.633314 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633179 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg9b2\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-kube-api-access-pg9b2\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.633314 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633197 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac732629-79f4-4eea-95f4-f5e748d23c05-cert\") pod \"ingress-canary-grgfz\" (UID: \"ac732629-79f4-4eea-95f4-f5e748d23c05\") " pod="openshift-ingress-canary/ingress-canary-grgfz" Jul 02 21:31:41.633314 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633217 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkmsd\" (UniqueName: \"kubernetes.io/projected/33485243-8457-4ab8-83a2-3ad6cc1d98f9-kube-api-access-zkmsd\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.633314 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633241 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.633314 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633260 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33485243-8457-4ab8-83a2-3ad6cc1d98f9-tmp-dir\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.633527 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633322 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de89456c-8281-4f09-84b8-6b2102028ff0-ca-trust-extracted\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.633527 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.633337 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-bound-sa-token\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.733741 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.733715 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbvlc\" (UniqueName: \"kubernetes.io/projected/ac732629-79f4-4eea-95f4-f5e748d23c05-kube-api-access-jbvlc\") pod \"ingress-canary-grgfz\" (UID: \"ac732629-79f4-4eea-95f4-f5e748d23c05\") " pod="openshift-ingress-canary/ingress-canary-grgfz" Jul 02 21:31:41.733848 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.733747 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/701b2907-f252-40d2-a8f9-b22923946c39-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:41.733848 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.733768 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-installation-pull-secrets\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.733914 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.733900 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-image-registry-private-configuration\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.733951 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.733932 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:41.734073 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734055 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-trusted-ca\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.734131 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:41.734089 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:31:41.734131 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734116 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/33485243-8457-4ab8-83a2-3ad6cc1d98f9-config-volume\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.734225 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:41.734152 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert podName:701b2907-f252-40d2-a8f9-b22923946c39 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:42.234132968 +0000 UTC m=+36.683971584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-w7bjn" (UID: "701b2907-f252-40d2-a8f9-b22923946c39") : secret "networking-console-plugin-cert" not found Jul 02 21:31:41.734225 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734173 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-registry-certificates\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.734225 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734204 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg9b2\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-kube-api-access-pg9b2\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.734401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734233 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac732629-79f4-4eea-95f4-f5e748d23c05-cert\") pod \"ingress-canary-grgfz\" (UID: \"ac732629-79f4-4eea-95f4-f5e748d23c05\") " pod="openshift-ingress-canary/ingress-canary-grgfz" Jul 02 21:31:41.734401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734263 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkmsd\" (UniqueName: \"kubernetes.io/projected/33485243-8457-4ab8-83a2-3ad6cc1d98f9-kube-api-access-zkmsd\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.734401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734289 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.734401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734317 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33485243-8457-4ab8-83a2-3ad6cc1d98f9-tmp-dir\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.734401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734337 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de89456c-8281-4f09-84b8-6b2102028ff0-ca-trust-extracted\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.734401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734360 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-bound-sa-token\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.734705 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734405 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.734705 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:41.734533 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:31:41.734705 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:41.734545 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7568cffd97-p78mw: secret "image-registry-tls" not found Jul 02 21:31:41.734705 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:41.734586 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls podName:de89456c-8281-4f09-84b8-6b2102028ff0 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:42.234571656 +0000 UTC m=+36.684410273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls") pod "image-registry-7568cffd97-p78mw" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0") : secret "image-registry-tls" not found Jul 02 21:31:41.734705 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734634 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/701b2907-f252-40d2-a8f9-b22923946c39-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:41.734952 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:41.734721 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:31:41.734952 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:41.734772 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls podName:33485243-8457-4ab8-83a2-3ad6cc1d98f9 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:42.234754424 +0000 UTC m=+36.684593025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls") pod "dns-default-wvj7q" (UID: "33485243-8457-4ab8-83a2-3ad6cc1d98f9") : secret "dns-default-metrics-tls" not found Jul 02 21:31:41.734952 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734851 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/33485243-8457-4ab8-83a2-3ad6cc1d98f9-config-volume\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.734952 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.734863 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-registry-certificates\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.735308 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.735288 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33485243-8457-4ab8-83a2-3ad6cc1d98f9-tmp-dir\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.735418 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.735289 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de89456c-8281-4f09-84b8-6b2102028ff0-ca-trust-extracted\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.735418 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.735404 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-trusted-ca\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.738195 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.738176 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-installation-pull-secrets\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.738195 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.738181 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-image-registry-private-configuration\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.738345 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.738185 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac732629-79f4-4eea-95f4-f5e748d23c05-cert\") pod \"ingress-canary-grgfz\" (UID: \"ac732629-79f4-4eea-95f4-f5e748d23c05\") " pod="openshift-ingress-canary/ingress-canary-grgfz" Jul 02 21:31:41.742801 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.742773 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbvlc\" (UniqueName: \"kubernetes.io/projected/ac732629-79f4-4eea-95f4-f5e748d23c05-kube-api-access-jbvlc\") pod \"ingress-canary-grgfz\" (UID: \"ac732629-79f4-4eea-95f4-f5e748d23c05\") " pod="openshift-ingress-canary/ingress-canary-grgfz" Jul 02 21:31:41.743008 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.742979 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg9b2\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-kube-api-access-pg9b2\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.743658 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.743597 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-bound-sa-token\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:41.743758 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.743738 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkmsd\" (UniqueName: \"kubernetes.io/projected/33485243-8457-4ab8-83a2-3ad6cc1d98f9-kube-api-access-zkmsd\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:41.832243 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:41.832219 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-grgfz" Jul 02 21:31:42.002900 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.002820 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:31:42.003037 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.002821 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:31:42.003037 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.002831 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:31:42.005702 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.005679 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 21:31:42.005834 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.005709 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 21:31:42.005834 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.005772 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sm2np\"" Jul 02 21:31:42.006028 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.006008 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 21:31:42.006131 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.006054 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 21:31:42.006131 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.006078 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7x7kj\"" Jul 02 21:31:42.023431 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.023408 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-grgfz"] Jul 02 21:31:42.026684 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:31:42.026658 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac732629_79f4_4eea_95f4_f5e748d23c05.slice/crio-5b39de226e4851e3f9b9380345475ab8e376600a3299434a5329f1d596a2eb59 WatchSource:0}: Error finding container 5b39de226e4851e3f9b9380345475ab8e376600a3299434a5329f1d596a2eb59: Status 404 returned error can't find the container with id 5b39de226e4851e3f9b9380345475ab8e376600a3299434a5329f1d596a2eb59 Jul 02 21:31:42.239503 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.239469 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:42.239678 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.239518 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:42.239678 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.239543 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:42.239678 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:42.239609 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:31:42.239678 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:42.239668 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert podName:701b2907-f252-40d2-a8f9-b22923946c39 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:43.239654058 +0000 UTC m=+37.689492660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-w7bjn" (UID: "701b2907-f252-40d2-a8f9-b22923946c39") : secret "networking-console-plugin-cert" not found Jul 02 21:31:42.239885 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:42.239613 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:31:42.239885 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:42.239771 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls podName:33485243-8457-4ab8-83a2-3ad6cc1d98f9 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:43.239749763 +0000 UTC m=+37.689588391 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls") pod "dns-default-wvj7q" (UID: "33485243-8457-4ab8-83a2-3ad6cc1d98f9") : secret "dns-default-metrics-tls" not found Jul 02 21:31:42.239885 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:42.239614 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:31:42.239885 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:42.239793 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7568cffd97-p78mw: secret "image-registry-tls" not found Jul 02 21:31:42.239885 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:42.239840 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls podName:de89456c-8281-4f09-84b8-6b2102028ff0 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:43.239825202 +0000 UTC m=+37.689663818 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls") pod "image-registry-7568cffd97-p78mw" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0") : secret "image-registry-tls" not found Jul 02 21:31:42.418225 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.418184 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-grgfz" event={"ID":"ac732629-79f4-4eea-95f4-f5e748d23c05","Type":"ContainerStarted","Data":"5b39de226e4851e3f9b9380345475ab8e376600a3299434a5329f1d596a2eb59"} Jul 02 21:31:42.420852 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.420828 2547 generic.go:356] "Generic (PLEG): container finished" podID="b035aac5-e945-44dd-a4f6-131fca47ad48" containerID="d36b6b811a762028150603e9b9517d21e6d8a8a88631512f6bd80175a7da62ff" exitCode=0 Jul 02 21:31:42.420948 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:42.420866 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bswhr" event={"ID":"b035aac5-e945-44dd-a4f6-131fca47ad48","Type":"ContainerDied","Data":"d36b6b811a762028150603e9b9517d21e6d8a8a88631512f6bd80175a7da62ff"} Jul 02 21:31:43.248916 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:43.248688 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:43.249099 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:43.248972 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:43.249099 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:43.248844 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:31:43.249099 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:43.249010 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:43.249099 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:43.249066 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert podName:701b2907-f252-40d2-a8f9-b22923946c39 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:45.249042941 +0000 UTC m=+39.698881556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-w7bjn" (UID: "701b2907-f252-40d2-a8f9-b22923946c39") : secret "networking-console-plugin-cert" not found Jul 02 21:31:43.249332 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:43.249153 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:31:43.249332 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:43.249168 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7568cffd97-p78mw: secret "image-registry-tls" not found Jul 02 21:31:43.249332 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:43.249167 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:31:43.249332 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:43.249223 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls podName:de89456c-8281-4f09-84b8-6b2102028ff0 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:45.249207797 +0000 UTC m=+39.699046404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls") pod "image-registry-7568cffd97-p78mw" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0") : secret "image-registry-tls" not found Jul 02 21:31:43.249332 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:43.249241 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls podName:33485243-8457-4ab8-83a2-3ad6cc1d98f9 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:45.249231795 +0000 UTC m=+39.699070403 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls") pod "dns-default-wvj7q" (UID: "33485243-8457-4ab8-83a2-3ad6cc1d98f9") : secret "dns-default-metrics-tls" not found Jul 02 21:31:43.428210 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:43.428172 2547 generic.go:356] "Generic (PLEG): container finished" podID="b035aac5-e945-44dd-a4f6-131fca47ad48" containerID="f37aa07120b1128529d8dc3afa839eaee48a12bc3cf439ee863f8f63312196a1" exitCode=0 Jul 02 21:31:43.428703 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:43.428230 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bswhr" event={"ID":"b035aac5-e945-44dd-a4f6-131fca47ad48","Type":"ContainerDied","Data":"f37aa07120b1128529d8dc3afa839eaee48a12bc3cf439ee863f8f63312196a1"} Jul 02 21:31:44.434925 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:44.434889 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bswhr" event={"ID":"b035aac5-e945-44dd-a4f6-131fca47ad48","Type":"ContainerStarted","Data":"594cb8852f89218eeb51320b04db89123dd150913f5bcab9290ca1c298ead32f"} Jul 02 21:31:44.459113 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:44.459060 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bswhr" podStartSLOduration=8.390582729 podStartE2EDuration="38.459041619s" totalImagesPullingTime="30.06845889s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:44.457080329 +0000 UTC m=+38.906918954" watchObservedRunningTime="2026-07-02 21:31:44.459041619 +0000 UTC m=+38.908880243" Jul 02 21:31:45.268935 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:45.268894 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:45.269114 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:45.268962 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:45.269114 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:45.269068 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:31:45.269238 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:45.269139 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls podName:33485243-8457-4ab8-83a2-3ad6cc1d98f9 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:49.269120421 +0000 UTC m=+43.718959042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls") pod "dns-default-wvj7q" (UID: "33485243-8457-4ab8-83a2-3ad6cc1d98f9") : secret "dns-default-metrics-tls" not found Jul 02 21:31:45.269238 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:45.269077 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:31:45.269238 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:45.269170 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7568cffd97-p78mw: secret "image-registry-tls" not found Jul 02 21:31:45.269238 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:45.269209 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:45.269238 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:45.269224 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls podName:de89456c-8281-4f09-84b8-6b2102028ff0 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:49.269207812 +0000 UTC m=+43.719046429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls") pod "image-registry-7568cffd97-p78mw" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0") : secret "image-registry-tls" not found Jul 02 21:31:45.269441 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:45.269272 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:31:45.269441 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:45.269306 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert podName:701b2907-f252-40d2-a8f9-b22923946c39 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:49.269295656 +0000 UTC m=+43.719134272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-w7bjn" (UID: "701b2907-f252-40d2-a8f9-b22923946c39") : secret "networking-console-plugin-cert" not found Jul 02 21:31:45.438878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:45.438787 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-grgfz" event={"ID":"ac732629-79f4-4eea-95f4-f5e748d23c05","Type":"ContainerStarted","Data":"3a34e80bd6b08ee91787689b049f6f1084698413f4ab63b4210cea65630035b2"} Jul 02 21:31:45.454598 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:45.454543 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-grgfz" podStartSLOduration=1.365829331 podStartE2EDuration="4.454526327s" totalImagesPullingTime="3.088696996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:31:45.454420643 +0000 UTC m=+39.904259264" watchObservedRunningTime="2026-07-02 21:31:45.454526327 +0000 UTC m=+39.904364952" Jul 02 21:31:49.299146 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:49.299100 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:49.299599 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:49.299159 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:49.299599 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:49.299181 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:49.299599 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:49.299261 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:31:49.299599 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:49.299279 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:31:49.299599 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:49.299289 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7568cffd97-p78mw: secret "image-registry-tls" not found Jul 02 21:31:49.299599 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:49.299305 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:31:49.299599 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:49.299341 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert podName:701b2907-f252-40d2-a8f9-b22923946c39 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:57.299320331 +0000 UTC m=+51.749158945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-w7bjn" (UID: "701b2907-f252-40d2-a8f9-b22923946c39") : secret "networking-console-plugin-cert" not found Jul 02 21:31:49.299599 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:49.299357 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls podName:33485243-8457-4ab8-83a2-3ad6cc1d98f9 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:57.299351141 +0000 UTC m=+51.749189742 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls") pod "dns-default-wvj7q" (UID: "33485243-8457-4ab8-83a2-3ad6cc1d98f9") : secret "dns-default-metrics-tls" not found Jul 02 21:31:49.299599 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:49.299370 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls podName:de89456c-8281-4f09-84b8-6b2102028ff0 nodeName:}" failed. No retries permitted until 2026-07-02 21:31:57.299364428 +0000 UTC m=+51.749203066 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls") pod "image-registry-7568cffd97-p78mw" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0") : secret "image-registry-tls" not found Jul 02 21:31:57.356138 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:57.356085 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:31:57.356707 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:57.356175 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:31:57.356707 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:31:57.356210 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:31:57.356707 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:57.356227 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:31:57.356707 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:57.356291 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert podName:701b2907-f252-40d2-a8f9-b22923946c39 nodeName:}" failed. No retries permitted until 2026-07-02 21:32:13.356275118 +0000 UTC m=+67.806113720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-w7bjn" (UID: "701b2907-f252-40d2-a8f9-b22923946c39") : secret "networking-console-plugin-cert" not found Jul 02 21:31:57.356707 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:57.356306 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:31:57.356707 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:57.356340 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:31:57.356707 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:57.356353 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7568cffd97-p78mw: secret "image-registry-tls" not found Jul 02 21:31:57.356707 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:57.356368 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls podName:33485243-8457-4ab8-83a2-3ad6cc1d98f9 nodeName:}" failed. No retries permitted until 2026-07-02 21:32:13.356355731 +0000 UTC m=+67.806194332 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls") pod "dns-default-wvj7q" (UID: "33485243-8457-4ab8-83a2-3ad6cc1d98f9") : secret "dns-default-metrics-tls" not found Jul 02 21:31:57.356707 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:31:57.356429 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls podName:de89456c-8281-4f09-84b8-6b2102028ff0 nodeName:}" failed. No retries permitted until 2026-07-02 21:32:13.356412873 +0000 UTC m=+67.806251490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls") pod "image-registry-7568cffd97-p78mw" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0") : secret "image-registry-tls" not found Jul 02 21:32:06.410194 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:06.410165 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-84htm" Jul 02 21:32:10.647661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.647614 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:32:10.650511 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.650492 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 21:32:10.661346 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.661314 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6c4320e-5777-4358-8dac-d43f1f4f411e-original-pull-secret\") pod \"global-pull-secret-syncer-hsd7b\" (UID: \"e6c4320e-5777-4358-8dac-d43f1f4f411e\") " pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:32:10.748240 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.748204 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:32:10.750833 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.750818 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 21:32:10.758915 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:10.758899 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 21:32:10.758971 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:10.758953 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:33:14.758938632 +0000 UTC m=+129.208777247 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : secret "metrics-daemon-secret" not found Jul 02 21:32:10.831525 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.831481 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hsd7b" Jul 02 21:32:10.849001 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.848969 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:32:10.851823 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.851795 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 21:32:10.862394 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.862359 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 21:32:10.872994 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.872968 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgj9v\" (UniqueName: \"kubernetes.io/projected/d4147e92-e8f2-4a19-8fe7-fdd1c8a52174-kube-api-access-wgj9v\") pod \"network-check-target-pqqpn\" (UID: \"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174\") " pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:32:10.958500 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:10.958472 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hsd7b"] Jul 02 21:32:10.961662 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:32:10.961640 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6c4320e_5777_4358_8dac_d43f1f4f411e.slice/crio-13aeaafd9eb21407c2f10905c1527bc04e581a2c14568b32f61aca0669f37cc5 WatchSource:0}: Error finding container 13aeaafd9eb21407c2f10905c1527bc04e581a2c14568b32f61aca0669f37cc5: Status 404 returned error can't find the container with id 13aeaafd9eb21407c2f10905c1527bc04e581a2c14568b32f61aca0669f37cc5 Jul 02 21:32:11.145182 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:11.145153 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sm2np\"" Jul 02 21:32:11.152661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:11.152632 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:32:11.292655 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:11.292625 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pqqpn"] Jul 02 21:32:11.295751 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:32:11.295727 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4147e92_e8f2_4a19_8fe7_fdd1c8a52174.slice/crio-25e04feb0d19d4b5cc81fce28cf81834bfdb6ba8dde488bde3e4a9efa8fca606 WatchSource:0}: Error finding container 25e04feb0d19d4b5cc81fce28cf81834bfdb6ba8dde488bde3e4a9efa8fca606: Status 404 returned error can't find the container with id 25e04feb0d19d4b5cc81fce28cf81834bfdb6ba8dde488bde3e4a9efa8fca606 Jul 02 21:32:11.498160 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:11.498064 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pqqpn" event={"ID":"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174","Type":"ContainerStarted","Data":"25e04feb0d19d4b5cc81fce28cf81834bfdb6ba8dde488bde3e4a9efa8fca606"} Jul 02 21:32:11.499128 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:11.499107 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hsd7b" event={"ID":"e6c4320e-5777-4358-8dac-d43f1f4f411e","Type":"ContainerStarted","Data":"13aeaafd9eb21407c2f10905c1527bc04e581a2c14568b32f61aca0669f37cc5"} Jul 02 21:32:13.371124 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:13.371072 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:32:13.371610 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:13.371177 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:32:13.371610 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:13.371220 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:32:13.371610 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:13.371266 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:32:13.371610 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:13.371294 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7568cffd97-p78mw: secret "image-registry-tls" not found Jul 02 21:32:13.371610 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:13.371321 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:32:13.371610 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:13.371324 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:32:13.371610 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:13.371399 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls podName:de89456c-8281-4f09-84b8-6b2102028ff0 nodeName:}" failed. No retries permitted until 2026-07-02 21:32:45.371359058 +0000 UTC m=+99.821197660 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls") pod "image-registry-7568cffd97-p78mw" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0") : secret "image-registry-tls" not found Jul 02 21:32:13.371610 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:13.371422 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls podName:33485243-8457-4ab8-83a2-3ad6cc1d98f9 nodeName:}" failed. No retries permitted until 2026-07-02 21:32:45.371410877 +0000 UTC m=+99.821249484 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls") pod "dns-default-wvj7q" (UID: "33485243-8457-4ab8-83a2-3ad6cc1d98f9") : secret "dns-default-metrics-tls" not found Jul 02 21:32:13.371610 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:13.371440 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert podName:701b2907-f252-40d2-a8f9-b22923946c39 nodeName:}" failed. No retries permitted until 2026-07-02 21:32:45.371429657 +0000 UTC m=+99.821268258 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-w7bjn" (UID: "701b2907-f252-40d2-a8f9-b22923946c39") : secret "networking-console-plugin-cert" not found Jul 02 21:32:16.515251 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:16.515158 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pqqpn" event={"ID":"d4147e92-e8f2-4a19-8fe7-fdd1c8a52174","Type":"ContainerStarted","Data":"f101b7402bef7741b2c61ec9c9b0553dfc716a09aa92d83dc3ffb7653d0dbda5"} Jul 02 21:32:16.515717 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:16.515274 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:32:16.516659 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:16.516635 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hsd7b" event={"ID":"e6c4320e-5777-4358-8dac-d43f1f4f411e","Type":"ContainerStarted","Data":"76bf3ef22d79b6439f210f2c9608b9e0af616babc8f28c114a947d57b121f830"} Jul 02 21:32:16.535469 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:16.534164 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pqqpn" podStartSLOduration=65.766799939 podStartE2EDuration="1m10.534144678s" totalImagesPullingTime="4.767344739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:32:16.533262551 +0000 UTC m=+70.983101174" watchObservedRunningTime="2026-07-02 21:32:16.534144678 +0000 UTC m=+70.983983338" Jul 02 21:32:16.551524 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:16.551475 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hsd7b" podStartSLOduration=65.439462738 podStartE2EDuration="1m10.551457951s" totalImagesPullingTime="5.111995213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:32:16.55121152 +0000 UTC m=+71.001050145" watchObservedRunningTime="2026-07-02 21:32:16.551457951 +0000 UTC m=+71.001296567" Jul 02 21:32:45.399933 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:45.399889 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:32:45.400465 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:45.399946 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:32:45.400465 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:45.399969 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:32:45.400465 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:45.400039 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:32:45.400465 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:45.400052 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:32:45.400465 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:45.400057 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:32:45.400465 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:45.400112 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert podName:701b2907-f252-40d2-a8f9-b22923946c39 nodeName:}" failed. No retries permitted until 2026-07-02 21:33:49.400098744 +0000 UTC m=+163.849937359 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-w7bjn" (UID: "701b2907-f252-40d2-a8f9-b22923946c39") : secret "networking-console-plugin-cert" not found Jul 02 21:32:45.400465 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:45.400141 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls podName:33485243-8457-4ab8-83a2-3ad6cc1d98f9 nodeName:}" failed. No retries permitted until 2026-07-02 21:33:49.400122339 +0000 UTC m=+163.849960958 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls") pod "dns-default-wvj7q" (UID: "33485243-8457-4ab8-83a2-3ad6cc1d98f9") : secret "dns-default-metrics-tls" not found Jul 02 21:32:45.400465 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:45.400064 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7568cffd97-p78mw: secret "image-registry-tls" not found Jul 02 21:32:45.400465 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:32:45.400197 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls podName:de89456c-8281-4f09-84b8-6b2102028ff0 nodeName:}" failed. No retries permitted until 2026-07-02 21:33:49.400186071 +0000 UTC m=+163.850024673 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls") pod "image-registry-7568cffd97-p78mw" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0") : secret "image-registry-tls" not found Jul 02 21:32:47.522189 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:32:47.522149 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pqqpn" Jul 02 21:33:14.807433 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:33:14.807362 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:33:14.807919 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:14.807512 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 21:33:14.807919 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:14.807583 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:35:16.807567472 +0000 UTC m=+251.257406091 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : secret "metrics-daemon-secret" not found Jul 02 21:33:44.478941 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:44.478888 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" podUID="de89456c-8281-4f09-84b8-6b2102028ff0" Jul 02 21:33:44.505571 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:44.505540 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" podUID="701b2907-f252-40d2-a8f9-b22923946c39" Jul 02 21:33:44.517885 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:44.517857 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wvj7q" podUID="33485243-8457-4ab8-83a2-3ad6cc1d98f9" Jul 02 21:33:44.722525 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:33:44.722493 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvj7q" Jul 02 21:33:44.722678 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:33:44.722496 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:33:44.722678 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:33:44.722496 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:33:45.037203 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:45.037152 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:33:49.453236 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:33:49.453196 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:33:49.453783 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:33:49.453247 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:33:49.453783 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:33:49.453269 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:33:49.453783 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:49.453367 2547 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 21:33:49.453783 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:49.453478 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls podName:33485243-8457-4ab8-83a2-3ad6cc1d98f9 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:51.453461459 +0000 UTC m=+285.903300077 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls") pod "dns-default-wvj7q" (UID: "33485243-8457-4ab8-83a2-3ad6cc1d98f9") : secret "dns-default-metrics-tls" not found Jul 02 21:33:49.453783 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:49.453369 2547 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 21:33:49.453783 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:49.453523 2547 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7568cffd97-p78mw: secret "image-registry-tls" not found Jul 02 21:33:49.453783 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:49.453369 2547 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 02 21:33:49.453783 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:49.453593 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls podName:de89456c-8281-4f09-84b8-6b2102028ff0 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:51.45357518 +0000 UTC m=+285.903413800 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls") pod "image-registry-7568cffd97-p78mw" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0") : secret "image-registry-tls" not found Jul 02 21:33:49.453783 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:33:49.453624 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert podName:701b2907-f252-40d2-a8f9-b22923946c39 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:51.453613384 +0000 UTC m=+285.903451986 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-w7bjn" (UID: "701b2907-f252-40d2-a8f9-b22923946c39") : secret "networking-console-plugin-cert" not found Jul 02 21:33:59.002608 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:33:59.002511 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:35:16.873637 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:16.873596 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:35:16.874112 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:16.873746 2547 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 21:35:16.874112 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:16.873811 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs podName:f4f1ee0d-0554-4134-b502-08c8aefca7ec nodeName:}" failed. No retries permitted until 2026-07-02 21:37:18.873795713 +0000 UTC m=+373.323634315 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs") pod "network-metrics-daemon-95lhp" (UID: "f4f1ee0d-0554-4134-b502-08c8aefca7ec") : secret "metrics-daemon-secret" not found Jul 02 21:35:19.423564 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.423535 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6"] Jul 02 21:35:19.426271 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.426255 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6" Jul 02 21:35:19.428809 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.428785 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 02 21:35:19.428980 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.428882 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:35:19.429845 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.429822 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-h8hhg\"" Jul 02 21:35:19.431670 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.431647 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd"] Jul 02 21:35:19.434549 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.434529 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-j6j5h"] Jul 02 21:35:19.434650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.434635 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:19.437075 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.437058 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w"] Jul 02 21:35:19.437185 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.437168 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.437737 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.437712 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 02 21:35:19.439001 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.438979 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xdkxp\"" Jul 02 21:35:19.439098 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.439009 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 02 21:35:19.439098 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.439020 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 21:35:19.439098 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.439011 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 21:35:19.439945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.439911 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:19.439945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.439924 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6"] Jul 02 21:35:19.440148 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.440013 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:35:19.440148 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.440029 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 02 21:35:19.440372 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.440353 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 02 21:35:19.440554 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.440538 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-lknf5\"" Jul 02 21:35:19.440634 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.440551 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 02 21:35:19.443056 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.443037 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 02 21:35:19.443171 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.443132 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:35:19.443412 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.443372 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 02 21:35:19.443809 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.443791 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-xv6tv\"" Jul 02 21:35:19.446983 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.446965 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 02 21:35:19.460341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.460318 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd"] Jul 02 21:35:19.463458 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.463435 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-j6j5h"] Jul 02 21:35:19.464508 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.464488 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w"] Jul 02 21:35:19.495334 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495311 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpmnr\" (UniqueName: \"kubernetes.io/projected/64ca734d-9b7d-4802-9462-e5456cb7fb3a-kube-api-access-zpmnr\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:19.495508 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495338 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e011139-283a-47f3-8490-1011cd239396-trusted-ca\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.495508 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495399 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/64ca734d-9b7d-4802-9462-e5456cb7fb3a-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:19.495508 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495415 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:19.495508 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495484 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48gl4\" (UniqueName: \"kubernetes.io/projected/6e011139-283a-47f3-8490-1011cd239396-kube-api-access-48gl4\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.495703 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495511 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grg4f\" (UniqueName: \"kubernetes.io/projected/b4ac7946-f208-4b2b-9462-3916a0a72e8e-kube-api-access-grg4f\") pod \"volume-data-source-validator-5d456fdbcd-lg2n6\" (UID: \"b4ac7946-f208-4b2b-9462-3916a0a72e8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6" Jul 02 21:35:19.495703 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495532 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e011139-283a-47f3-8490-1011cd239396-config\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.495703 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495548 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e011139-283a-47f3-8490-1011cd239396-serving-cert\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.495703 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495568 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:19.495703 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.495626 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6fpq\" (UniqueName: \"kubernetes.io/projected/076c9fc6-3433-47b4-be0e-b09a15866344-kube-api-access-l6fpq\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:19.527842 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.527809 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7bc4c77f4f-2bbsx"] Jul 02 21:35:19.530713 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.530698 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.535425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.535350 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 02 21:35:19.535425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.535337 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 02 21:35:19.535602 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.535543 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7ldvv\"" Jul 02 21:35:19.535853 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.535832 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 02 21:35:19.535853 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.535845 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 21:35:19.536014 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.535834 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 21:35:19.536150 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.536136 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 02 21:35:19.547711 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.547678 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7bc4c77f4f-2bbsx"] Jul 02 21:35:19.596878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.596850 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grg4f\" (UniqueName: \"kubernetes.io/projected/b4ac7946-f208-4b2b-9462-3916a0a72e8e-kube-api-access-grg4f\") pod \"volume-data-source-validator-5d456fdbcd-lg2n6\" (UID: \"b4ac7946-f208-4b2b-9462-3916a0a72e8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6" Jul 02 21:35:19.597031 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.596895 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmnx4\" (UniqueName: \"kubernetes.io/projected/5aa921ab-8003-4f10-83ea-9842fb74856a-kube-api-access-tmnx4\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.597031 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.596919 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e011139-283a-47f3-8490-1011cd239396-config\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.597107 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597090 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e011139-283a-47f3-8490-1011cd239396-serving-cert\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.597139 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597124 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.597192 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597146 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-default-certificate\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.597249 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597228 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpmnr\" (UniqueName: \"kubernetes.io/projected/64ca734d-9b7d-4802-9462-e5456cb7fb3a-kube-api-access-zpmnr\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:19.597298 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597287 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/64ca734d-9b7d-4802-9462-e5456cb7fb3a-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:19.597333 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597311 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48gl4\" (UniqueName: \"kubernetes.io/projected/6e011139-283a-47f3-8490-1011cd239396-kube-api-access-48gl4\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.597444 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597426 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.597516 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597470 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-stats-auth\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.597516 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597508 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:19.597769 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597536 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6e011139-283a-47f3-8490-1011cd239396-config\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.597769 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597539 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e011139-283a-47f3-8490-1011cd239396-trusted-ca\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.597769 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:19.597589 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:19.597769 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597601 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6fpq\" (UniqueName: \"kubernetes.io/projected/076c9fc6-3433-47b4-be0e-b09a15866344-kube-api-access-l6fpq\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:19.597769 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.597646 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:19.597769 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:19.597664 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls podName:64ca734d-9b7d-4802-9462-e5456cb7fb3a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:20.097645258 +0000 UTC m=+254.547483874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xb5nd" (UID: "64ca734d-9b7d-4802-9462-e5456cb7fb3a") : secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:19.597769 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:19.597767 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 21:35:19.598124 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:19.597819 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls podName:076c9fc6-3433-47b4-be0e-b09a15866344 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:20.097803774 +0000 UTC m=+254.547642389 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-kv27w" (UID: "076c9fc6-3433-47b4-be0e-b09a15866344") : secret "samples-operator-tls" not found Jul 02 21:35:19.598638 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.598614 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/64ca734d-9b7d-4802-9462-e5456cb7fb3a-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:19.598729 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.598641 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e011139-283a-47f3-8490-1011cd239396-trusted-ca\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.599394 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.599363 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6e011139-283a-47f3-8490-1011cd239396-serving-cert\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.605849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.605828 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48gl4\" (UniqueName: \"kubernetes.io/projected/6e011139-283a-47f3-8490-1011cd239396-kube-api-access-48gl4\") pod \"console-operator-68b757fb67-j6j5h\" (UID: \"6e011139-283a-47f3-8490-1011cd239396\") " pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.605947 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.605928 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6fpq\" (UniqueName: \"kubernetes.io/projected/076c9fc6-3433-47b4-be0e-b09a15866344-kube-api-access-l6fpq\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:19.606113 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.606092 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpmnr\" (UniqueName: \"kubernetes.io/projected/64ca734d-9b7d-4802-9462-e5456cb7fb3a-kube-api-access-zpmnr\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:19.606152 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.606137 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grg4f\" (UniqueName: \"kubernetes.io/projected/b4ac7946-f208-4b2b-9462-3916a0a72e8e-kube-api-access-grg4f\") pod \"volume-data-source-validator-5d456fdbcd-lg2n6\" (UID: \"b4ac7946-f208-4b2b-9462-3916a0a72e8e\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6" Jul 02 21:35:19.698849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.698752 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmnx4\" (UniqueName: \"kubernetes.io/projected/5aa921ab-8003-4f10-83ea-9842fb74856a-kube-api-access-tmnx4\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.698849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.698809 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.698849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.698851 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-default-certificate\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.699128 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:19.698938 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 21:35:19.699128 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.698945 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.699128 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:19.699017 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:20.198996362 +0000 UTC m=+254.648834979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : secret "router-metrics-certs-default" not found Jul 02 21:35:19.699128 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:19.699041 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:20.199031079 +0000 UTC m=+254.648869688 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : configmap references non-existent config key: service-ca.crt Jul 02 21:35:19.699128 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.699076 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-stats-auth\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.701264 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.701241 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-stats-auth\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.701424 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.701403 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-default-certificate\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.708725 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.708704 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmnx4\" (UniqueName: \"kubernetes.io/projected/5aa921ab-8003-4f10-83ea-9842fb74856a-kube-api-access-tmnx4\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:19.735733 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.735702 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6" Jul 02 21:35:19.750997 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.750969 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:19.864526 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.863814 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6"] Jul 02 21:35:19.868350 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:19.868321 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ac7946_f208_4b2b_9462_3916a0a72e8e.slice/crio-33edf3c22a6efc234b5f6f54133fa2ee2fc5aa2dcb4c480bf49a4c277f1fd7be WatchSource:0}: Error finding container 33edf3c22a6efc234b5f6f54133fa2ee2fc5aa2dcb4c480bf49a4c277f1fd7be: Status 404 returned error can't find the container with id 33edf3c22a6efc234b5f6f54133fa2ee2fc5aa2dcb4c480bf49a4c277f1fd7be Jul 02 21:35:19.882860 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.882833 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-j6j5h"] Jul 02 21:35:19.885370 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:19.885348 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e011139_283a_47f3_8490_1011cd239396.slice/crio-422e6a19767a122a53266ca6e3bffd920f95449a6403a0ec48b27b56d447095f WatchSource:0}: Error finding container 422e6a19767a122a53266ca6e3bffd920f95449a6403a0ec48b27b56d447095f: Status 404 returned error can't find the container with id 422e6a19767a122a53266ca6e3bffd920f95449a6403a0ec48b27b56d447095f Jul 02 21:35:19.934930 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.934896 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" event={"ID":"6e011139-283a-47f3-8490-1011cd239396","Type":"ContainerStarted","Data":"422e6a19767a122a53266ca6e3bffd920f95449a6403a0ec48b27b56d447095f"} Jul 02 21:35:19.935948 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:19.935927 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6" event={"ID":"b4ac7946-f208-4b2b-9462-3916a0a72e8e","Type":"ContainerStarted","Data":"33edf3c22a6efc234b5f6f54133fa2ee2fc5aa2dcb4c480bf49a4c277f1fd7be"} Jul 02 21:35:20.103453 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:20.103371 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:20.103639 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:20.103476 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:20.103639 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:20.103527 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:20.103639 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:20.103586 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 21:35:20.103639 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:20.103604 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls podName:64ca734d-9b7d-4802-9462-e5456cb7fb3a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:21.103586017 +0000 UTC m=+255.553424646 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xb5nd" (UID: "64ca734d-9b7d-4802-9462-e5456cb7fb3a") : secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:20.103639 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:20.103625 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls podName:076c9fc6-3433-47b4-be0e-b09a15866344 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:21.103614408 +0000 UTC m=+255.553453009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-kv27w" (UID: "076c9fc6-3433-47b4-be0e-b09a15866344") : secret "samples-operator-tls" not found Jul 02 21:35:20.204134 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:20.204095 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:20.204294 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:20.204169 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:20.204294 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:20.204276 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:21.204259055 +0000 UTC m=+255.654097673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : configmap references non-existent config key: service-ca.crt Jul 02 21:35:20.204294 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:20.204280 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 21:35:20.204472 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:20.204319 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:21.204306929 +0000 UTC m=+255.654145536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : secret "router-metrics-certs-default" not found Jul 02 21:35:21.111125 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:21.111085 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:21.111618 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:21.111270 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 21:35:21.111618 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:21.111342 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls podName:076c9fc6-3433-47b4-be0e-b09a15866344 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:23.111326625 +0000 UTC m=+257.561165227 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-kv27w" (UID: "076c9fc6-3433-47b4-be0e-b09a15866344") : secret "samples-operator-tls" not found Jul 02 21:35:21.111618 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:21.111402 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:21.111618 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:21.111513 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:21.111618 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:21.111557 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls podName:64ca734d-9b7d-4802-9462-e5456cb7fb3a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:23.111544509 +0000 UTC m=+257.561383124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xb5nd" (UID: "64ca734d-9b7d-4802-9462-e5456cb7fb3a") : secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:21.212530 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:21.212489 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:21.212745 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:21.212568 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:21.212745 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:21.212678 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 21:35:21.212745 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:21.212700 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:23.212683944 +0000 UTC m=+257.662522567 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : configmap references non-existent config key: service-ca.crt Jul 02 21:35:21.212745 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:21.212746 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:23.212728701 +0000 UTC m=+257.662567318 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : secret "router-metrics-certs-default" not found Jul 02 21:35:21.945019 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:21.944988 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6" event={"ID":"b4ac7946-f208-4b2b-9462-3916a0a72e8e","Type":"ContainerStarted","Data":"edea0195c926e9257b347f27888aa6846c3c20add5c7be32283e39e890ebc7bb"} Jul 02 21:35:21.961236 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:21.961160 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-lg2n6" podStartSLOduration=1.836028928 podStartE2EDuration="2.961143926s" totalImagesPullingTime="1.125114998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:21.960883951 +0000 UTC m=+256.410722586" watchObservedRunningTime="2026-07-02 21:35:21.961143926 +0000 UTC m=+256.410982551" Jul 02 21:35:22.950040 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:22.950011 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/0.log" Jul 02 21:35:22.950511 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:22.950052 2547 generic.go:356] "Generic (PLEG): container finished" podID="6e011139-283a-47f3-8490-1011cd239396" containerID="72b0a0047dfe402b06adf569c438faff835edc3d0f83b7b24a21dbe08a1ca4d7" exitCode=255 Jul 02 21:35:22.950511 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:22.950122 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" event={"ID":"6e011139-283a-47f3-8490-1011cd239396","Type":"ContainerDied","Data":"72b0a0047dfe402b06adf569c438faff835edc3d0f83b7b24a21dbe08a1ca4d7"} Jul 02 21:35:22.950511 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:22.950318 2547 scope.go:117] "RemoveContainer" containerID="72b0a0047dfe402b06adf569c438faff835edc3d0f83b7b24a21dbe08a1ca4d7" Jul 02 21:35:23.127006 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.126968 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:23.127006 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.127013 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:23.127246 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:23.127097 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 21:35:23.127246 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:23.127102 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:23.127246 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:23.127148 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls podName:076c9fc6-3433-47b4-be0e-b09a15866344 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:27.127135186 +0000 UTC m=+261.576973805 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-kv27w" (UID: "076c9fc6-3433-47b4-be0e-b09a15866344") : secret "samples-operator-tls" not found Jul 02 21:35:23.127246 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:23.127208 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls podName:64ca734d-9b7d-4802-9462-e5456cb7fb3a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:27.127201569 +0000 UTC m=+261.577040170 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xb5nd" (UID: "64ca734d-9b7d-4802-9462-e5456cb7fb3a") : secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:23.228254 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.228149 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:23.228254 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.228238 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:23.228479 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:23.228318 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 21:35:23.228479 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:23.228348 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:27.228333347 +0000 UTC m=+261.678171949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : configmap references non-existent config key: service-ca.crt Jul 02 21:35:23.228479 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:23.228421 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:27.228402787 +0000 UTC m=+261.678241406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : secret "router-metrics-certs-default" not found Jul 02 21:35:23.955251 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.955222 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/1.log" Jul 02 21:35:23.955661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.955618 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/0.log" Jul 02 21:35:23.955661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.955649 2547 generic.go:356] "Generic (PLEG): container finished" podID="6e011139-283a-47f3-8490-1011cd239396" containerID="191201acd7cc7f6064635484303970985dc8794bd5210235c546d11bf7b8a38e" exitCode=255 Jul 02 21:35:23.955772 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.955689 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" event={"ID":"6e011139-283a-47f3-8490-1011cd239396","Type":"ContainerDied","Data":"191201acd7cc7f6064635484303970985dc8794bd5210235c546d11bf7b8a38e"} Jul 02 21:35:23.955772 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.955739 2547 scope.go:117] "RemoveContainer" containerID="72b0a0047dfe402b06adf569c438faff835edc3d0f83b7b24a21dbe08a1ca4d7" Jul 02 21:35:23.955985 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:23.955967 2547 scope.go:117] "RemoveContainer" containerID="191201acd7cc7f6064635484303970985dc8794bd5210235c546d11bf7b8a38e" Jul 02 21:35:23.956167 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:23.956146 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-j6j5h_openshift-console-operator(6e011139-283a-47f3-8490-1011cd239396)\"" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" podUID="6e011139-283a-47f3-8490-1011cd239396" Jul 02 21:35:24.402633 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.402592 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2x644"] Jul 02 21:35:24.406962 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.406941 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.409241 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.409214 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 21:35:24.409241 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.409228 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 21:35:24.410400 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.410364 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-97hq6\"" Jul 02 21:35:24.410517 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.410427 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 21:35:24.410517 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.410434 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 21:35:24.415992 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.415963 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2x644"] Jul 02 21:35:24.539968 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.539934 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ca7ece0-b874-4113-88c2-11a0bf25a165-crio-socket\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.540123 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.540014 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.540123 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.540036 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55hj6\" (UniqueName: \"kubernetes.io/projected/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-api-access-55hj6\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.540123 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.540053 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ca7ece0-b874-4113-88c2-11a0bf25a165-data-volume\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.540225 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.540183 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.640856 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.640817 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.640856 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.640862 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ca7ece0-b874-4113-88c2-11a0bf25a165-crio-socket\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.641097 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.640928 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.641097 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.640953 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55hj6\" (UniqueName: \"kubernetes.io/projected/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-api-access-55hj6\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.641097 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.640980 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ca7ece0-b874-4113-88c2-11a0bf25a165-data-volume\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.641097 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:24.640953 2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:35:24.641097 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.641012 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ca7ece0-b874-4113-88c2-11a0bf25a165-crio-socket\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.641097 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:24.641061 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:35:24.641097 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:24.641087 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:25.141060601 +0000 UTC m=+259.590899216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : configmap "kube-rbac-proxy" not found Jul 02 21:35:24.641334 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:24.641114 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:25.141104024 +0000 UTC m=+259.590942628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : secret "insights-runtime-extractor-tls" not found Jul 02 21:35:24.641372 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.641334 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ca7ece0-b874-4113-88c2-11a0bf25a165-data-volume\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.650777 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.650746 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55hj6\" (UniqueName: \"kubernetes.io/projected/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-api-access-55hj6\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:24.959814 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.959789 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/1.log" Jul 02 21:35:24.960282 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:24.960213 2547 scope.go:117] "RemoveContainer" containerID="191201acd7cc7f6064635484303970985dc8794bd5210235c546d11bf7b8a38e" Jul 02 21:35:24.960440 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:24.960417 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-j6j5h_openshift-console-operator(6e011139-283a-47f3-8490-1011cd239396)\"" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" podUID="6e011139-283a-47f3-8490-1011cd239396" Jul 02 21:35:25.145880 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:25.145838 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:25.146091 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:25.145944 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:25.146091 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:25.145955 2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:35:25.146091 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:25.146019 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:26.146003102 +0000 UTC m=+260.595841707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : configmap "kube-rbac-proxy" not found Jul 02 21:35:25.146091 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:25.146076 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:35:25.146274 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:25.146152 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:26.146132579 +0000 UTC m=+260.595971201 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : secret "insights-runtime-extractor-tls" not found Jul 02 21:35:26.154433 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:26.154402 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:26.154802 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:26.154477 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:26.154802 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:26.154501 2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:35:26.154802 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:26.154560 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:28.154543778 +0000 UTC m=+262.604382383 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : configmap "kube-rbac-proxy" not found Jul 02 21:35:26.154802 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:26.154583 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:35:26.154802 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:26.154626 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:28.154610641 +0000 UTC m=+262.604449243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : secret "insights-runtime-extractor-tls" not found Jul 02 21:35:26.195325 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:26.195301 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l2nsz_25b068f3-c4ca-4d18-b74c-1b8d653d27c8/dns-node-resolver/0.log" Jul 02 21:35:27.163019 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:27.162922 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:27.163019 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:27.162987 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:27.163459 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:27.163086 2547 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:27.163459 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:27.163165 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls podName:64ca734d-9b7d-4802-9462-e5456cb7fb3a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:35.163140512 +0000 UTC m=+269.612979114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-xb5nd" (UID: "64ca734d-9b7d-4802-9462-e5456cb7fb3a") : secret "cluster-monitoring-operator-tls" not found Jul 02 21:35:27.163459 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:27.163086 2547 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 02 21:35:27.163459 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:27.163218 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls podName:076c9fc6-3433-47b4-be0e-b09a15866344 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:35.163205154 +0000 UTC m=+269.613043776 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-kv27w" (UID: "076c9fc6-3433-47b4-be0e-b09a15866344") : secret "samples-operator-tls" not found Jul 02 21:35:27.264238 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:27.264202 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:27.264425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:27.264311 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:27.264425 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:27.264398 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:35.264363298 +0000 UTC m=+269.714201904 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : configmap references non-existent config key: service-ca.crt Jul 02 21:35:27.264512 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:27.264471 2547 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 02 21:35:27.264546 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:27.264525 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs podName:5aa921ab-8003-4f10-83ea-9842fb74856a nodeName:}" failed. No retries permitted until 2026-07-02 21:35:35.264511193 +0000 UTC m=+269.714349809 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs") pod "router-default-7bc4c77f4f-2bbsx" (UID: "5aa921ab-8003-4f10-83ea-9842fb74856a") : secret "router-metrics-certs-default" not found Jul 02 21:35:27.395480 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:27.395454 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gjvt4_47e464a7-c6bb-45fa-89b9-a4c5eb701478/node-ca/0.log" Jul 02 21:35:27.995971 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:27.995940 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-grgfz_ac732629-79f4-4eea-95f4-f5e748d23c05/serve-healthcheck-canary/0.log" Jul 02 21:35:28.171990 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:28.171948 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:28.172482 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:28.172088 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:28.172482 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:28.172317 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:35:28.172482 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:28.172413 2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:35:28.172482 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:28.172471 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:32.172432922 +0000 UTC m=+266.622271527 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : secret "insights-runtime-extractor-tls" not found Jul 02 21:35:28.172678 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:28.172502 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:32.172490707 +0000 UTC m=+266.622329323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : configmap "kube-rbac-proxy" not found Jul 02 21:35:29.751799 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:29.751715 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:29.751799 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:29.751797 2547 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:29.752168 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:29.752109 2547 scope.go:117] "RemoveContainer" containerID="191201acd7cc7f6064635484303970985dc8794bd5210235c546d11bf7b8a38e" Jul 02 21:35:29.752271 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:29.752250 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-j6j5h_openshift-console-operator(6e011139-283a-47f3-8490-1011cd239396)\"" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" podUID="6e011139-283a-47f3-8490-1011cd239396" Jul 02 21:35:32.205367 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:32.205317 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:32.205816 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:32.205442 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:32.205816 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:32.205477 2547 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 21:35:32.205816 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:32.205532 2547 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 21:35:32.205816 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:32.205542 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:40.205525699 +0000 UTC m=+274.655364318 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : secret "insights-runtime-extractor-tls" not found Jul 02 21:35:32.205816 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:32.205583 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm podName:6ca7ece0-b874-4113-88c2-11a0bf25a165 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:40.205571715 +0000 UTC m=+274.655410317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2x644" (UID: "6ca7ece0-b874-4113-88c2-11a0bf25a165") : configmap "kube-rbac-proxy" not found Jul 02 21:35:35.233119 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.233081 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:35.233521 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.233133 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:35.235467 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.235446 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64ca734d-9b7d-4802-9462-e5456cb7fb3a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-xb5nd\" (UID: \"64ca734d-9b7d-4802-9462-e5456cb7fb3a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:35.235633 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.235611 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/076c9fc6-3433-47b4-be0e-b09a15866344-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-kv27w\" (UID: \"076c9fc6-3433-47b4-be0e-b09a15866344\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:35.334083 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.334053 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:35.334186 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.334116 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:35.334618 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.334595 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5aa921ab-8003-4f10-83ea-9842fb74856a-service-ca-bundle\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:35.336086 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.336070 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5aa921ab-8003-4f10-83ea-9842fb74856a-metrics-certs\") pod \"router-default-7bc4c77f4f-2bbsx\" (UID: \"5aa921ab-8003-4f10-83ea-9842fb74856a\") " pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:35.344907 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.344879 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" Jul 02 21:35:35.356037 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.356016 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" Jul 02 21:35:35.438914 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.438883 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:35.477761 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.477729 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd"] Jul 02 21:35:35.493034 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.493010 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w"] Jul 02 21:35:35.572224 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.572188 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7bc4c77f4f-2bbsx"] Jul 02 21:35:35.575045 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:35.575020 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aa921ab_8003_4f10_83ea_9842fb74856a.slice/crio-ee3549669aa422be6e67b533a5bd5c3387f8a4bbbbf485cdd7e3090465f50d7e WatchSource:0}: Error finding container ee3549669aa422be6e67b533a5bd5c3387f8a4bbbbf485cdd7e3090465f50d7e: Status 404 returned error can't find the container with id ee3549669aa422be6e67b533a5bd5c3387f8a4bbbbf485cdd7e3090465f50d7e Jul 02 21:35:35.991548 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.991497 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" event={"ID":"64ca734d-9b7d-4802-9462-e5456cb7fb3a","Type":"ContainerStarted","Data":"e1181bafef2cab0cfd4807137c3a9390a04e6e0028de15635ee65bbde57022d7"} Jul 02 21:35:35.993181 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.993154 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" event={"ID":"5aa921ab-8003-4f10-83ea-9842fb74856a","Type":"ContainerStarted","Data":"a78c0718bf8dd8611248fbffa0797f0aea02d2a4ded3f7213bacc0f1186c6353"} Jul 02 21:35:35.993314 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.993199 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" event={"ID":"5aa921ab-8003-4f10-83ea-9842fb74856a","Type":"ContainerStarted","Data":"ee3549669aa422be6e67b533a5bd5c3387f8a4bbbbf485cdd7e3090465f50d7e"} Jul 02 21:35:35.994331 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:35.994307 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" event={"ID":"076c9fc6-3433-47b4-be0e-b09a15866344","Type":"ContainerStarted","Data":"fd0d61163d91612e1fb230468b2f55f79ee33245acdb03b97452d628dbe26e92"} Jul 02 21:35:36.012943 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:36.012888 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" podStartSLOduration=17.012864186 podStartE2EDuration="17.012864186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:35:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:36.011818725 +0000 UTC m=+270.461657357" watchObservedRunningTime="2026-07-02 21:35:36.012864186 +0000 UTC m=+270.462702811" Jul 02 21:35:36.439554 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:36.439518 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:36.442498 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:36.442464 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:36.999066 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:36.999035 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:37.000631 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:37.000606 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7bc4c77f4f-2bbsx" Jul 02 21:35:38.006214 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:38.006182 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" event={"ID":"64ca734d-9b7d-4802-9462-e5456cb7fb3a","Type":"ContainerStarted","Data":"fb488c0f822f2d7f67d7112b80a5debe8ae7e37d267d8aab4d14d1b027e79921"} Jul 02 21:35:38.006686 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:38.006228 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" event={"ID":"076c9fc6-3433-47b4-be0e-b09a15866344","Type":"ContainerStarted","Data":"9bd5ae7634de0bfec34ea62fdf4a52dbe2b1de89dd33ab9e337dd92dc4340cf8"} Jul 02 21:35:38.006686 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:38.006247 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" event={"ID":"076c9fc6-3433-47b4-be0e-b09a15866344","Type":"ContainerStarted","Data":"c586d309d7bca052499a3102904050f7ed6ffedfc4f28bb2e11bd8e002eba9c8"} Jul 02 21:35:38.019120 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:38.019081 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-xb5nd" podStartSLOduration=17.046104236 podStartE2EDuration="19.019067167s" totalImagesPullingTime="1.972962931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:38.018174605 +0000 UTC m=+272.468013229" watchObservedRunningTime="2026-07-02 21:35:38.019067167 +0000 UTC m=+272.468905790" Jul 02 21:35:38.032917 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:38.032862 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-kv27w" podStartSLOduration=17.128450229 podStartE2EDuration="19.032846447s" totalImagesPullingTime="1.904396218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:38.032327656 +0000 UTC m=+272.482166281" watchObservedRunningTime="2026-07-02 21:35:38.032846447 +0000 UTC m=+272.482685070" Jul 02 21:35:40.276029 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:40.275985 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:40.276448 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:40.276085 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:40.276671 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:40.276651 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ca7ece0-b874-4113-88c2-11a0bf25a165-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:40.278321 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:40.278302 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ca7ece0-b874-4113-88c2-11a0bf25a165-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2x644\" (UID: \"6ca7ece0-b874-4113-88c2-11a0bf25a165\") " pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:40.316525 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:40.316493 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2x644" Jul 02 21:35:40.442128 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:40.442095 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2x644"] Jul 02 21:35:40.445833 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:40.445787 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ca7ece0_b874_4113_88c2_11a0bf25a165.slice/crio-69fec0ac5dbd3965cf9dd14f613a60238e6403489f411fe020bb5c7140e6f205 WatchSource:0}: Error finding container 69fec0ac5dbd3965cf9dd14f613a60238e6403489f411fe020bb5c7140e6f205: Status 404 returned error can't find the container with id 69fec0ac5dbd3965cf9dd14f613a60238e6403489f411fe020bb5c7140e6f205 Jul 02 21:35:41.002732 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:41.002695 2547 scope.go:117] "RemoveContainer" containerID="191201acd7cc7f6064635484303970985dc8794bd5210235c546d11bf7b8a38e" Jul 02 21:35:41.021839 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:41.021756 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2x644" event={"ID":"6ca7ece0-b874-4113-88c2-11a0bf25a165","Type":"ContainerStarted","Data":"69fec0ac5dbd3965cf9dd14f613a60238e6403489f411fe020bb5c7140e6f205"} Jul 02 21:35:42.029615 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:42.029583 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:35:42.030090 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:42.030066 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/1.log" Jul 02 21:35:42.030165 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:42.030120 2547 generic.go:356] "Generic (PLEG): container finished" podID="6e011139-283a-47f3-8490-1011cd239396" containerID="9cbafff3dda2c6ebef5faf08fb1f40d45231dfe1491eb8360029a1368dede749" exitCode=255 Jul 02 21:35:42.030222 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:42.030188 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" event={"ID":"6e011139-283a-47f3-8490-1011cd239396","Type":"ContainerDied","Data":"9cbafff3dda2c6ebef5faf08fb1f40d45231dfe1491eb8360029a1368dede749"} Jul 02 21:35:42.030278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:42.030256 2547 scope.go:117] "RemoveContainer" containerID="191201acd7cc7f6064635484303970985dc8794bd5210235c546d11bf7b8a38e" Jul 02 21:35:42.030689 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:42.030667 2547 scope.go:117] "RemoveContainer" containerID="9cbafff3dda2c6ebef5faf08fb1f40d45231dfe1491eb8360029a1368dede749" Jul 02 21:35:42.030880 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:42.030855 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-j6j5h_openshift-console-operator(6e011139-283a-47f3-8490-1011cd239396)\"" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" podUID="6e011139-283a-47f3-8490-1011cd239396" Jul 02 21:35:44.041125 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:44.041096 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:35:45.045599 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:45.045567 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2x644" event={"ID":"6ca7ece0-b874-4113-88c2-11a0bf25a165","Type":"ContainerStarted","Data":"ee2407aad6b63a408083731a8d87ce385169bc5863cd6f5e31d0518a39f2e56f"} Jul 02 21:35:45.045964 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:45.045623 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2x644" event={"ID":"6ca7ece0-b874-4113-88c2-11a0bf25a165","Type":"ContainerStarted","Data":"7a5ceeb5145f1bf0cedd52e25178743fd915f414cb524119362a4724de90e6d1"} Jul 02 21:35:47.723102 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:47.723061 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" podUID="701b2907-f252-40d2-a8f9-b22923946c39" Jul 02 21:35:47.723102 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:47.723061 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-wvj7q" podUID="33485243-8457-4ab8-83a2-3ad6cc1d98f9" Jul 02 21:35:47.723535 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:47.723060 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" podUID="de89456c-8281-4f09-84b8-6b2102028ff0" Jul 02 21:35:48.056791 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:48.056762 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:35:48.056981 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:48.056791 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2x644" event={"ID":"6ca7ece0-b874-4113-88c2-11a0bf25a165","Type":"ContainerStarted","Data":"f3bb07e39f205677ef17b693ade3900fab12043f1b25f24a65b7b3ebd58dd4bc"} Jul 02 21:35:48.056981 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:48.056909 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:35:48.057111 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:48.057048 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvj7q" Jul 02 21:35:48.075297 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:48.075257 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2x644" podStartSLOduration=17.533818491 podStartE2EDuration="24.075243239s" totalImagesPullingTime="6.541424748s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 21:35:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:48.074057745 +0000 UTC m=+282.523896370" watchObservedRunningTime="2026-07-02 21:35:48.075243239 +0000 UTC m=+282.525081863" Jul 02 21:35:49.751785 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.751750 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:49.752231 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.751799 2547 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:35:49.752231 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.752107 2547 scope.go:117] "RemoveContainer" containerID="9cbafff3dda2c6ebef5faf08fb1f40d45231dfe1491eb8360029a1368dede749" Jul 02 21:35:49.752300 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:49.752260 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-j6j5h_openshift-console-operator(6e011139-283a-47f3-8490-1011cd239396)\"" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" podUID="6e011139-283a-47f3-8490-1011cd239396" Jul 02 21:35:49.813559 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.813530 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47"] Jul 02 21:35:49.816833 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.816809 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" Jul 02 21:35:49.819816 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.819791 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 21:35:49.820020 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.819996 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mwp86\"" Jul 02 21:35:49.828310 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.828291 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47"] Jul 02 21:35:49.856519 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.856491 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2b04bf6c-d4fd-4686-88ce-7c896a9bf1e9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-28k47\" (UID: \"2b04bf6c-d4fd-4686-88ce-7c896a9bf1e9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" Jul 02 21:35:49.957910 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.957862 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2b04bf6c-d4fd-4686-88ce-7c896a9bf1e9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-28k47\" (UID: \"2b04bf6c-d4fd-4686-88ce-7c896a9bf1e9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" Jul 02 21:35:49.960207 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:49.960186 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2b04bf6c-d4fd-4686-88ce-7c896a9bf1e9-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-28k47\" (UID: \"2b04bf6c-d4fd-4686-88ce-7c896a9bf1e9\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" Jul 02 21:35:50.126042 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:50.126012 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" Jul 02 21:35:50.250488 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:50.250459 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47"] Jul 02 21:35:50.253309 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:50.253284 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b04bf6c_d4fd_4686_88ce_7c896a9bf1e9.slice/crio-b8adaafecda1e3358577d63fab492c0037869794aa0574464046df58e59d3d78 WatchSource:0}: Error finding container b8adaafecda1e3358577d63fab492c0037869794aa0574464046df58e59d3d78: Status 404 returned error can't find the container with id b8adaafecda1e3358577d63fab492c0037869794aa0574464046df58e59d3d78 Jul 02 21:35:51.072029 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.071994 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" event={"ID":"2b04bf6c-d4fd-4686-88ce-7c896a9bf1e9","Type":"ContainerStarted","Data":"b8adaafecda1e3358577d63fab492c0037869794aa0574464046df58e59d3d78"} Jul 02 21:35:51.467940 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.467908 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:35:51.468131 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.467979 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:35:51.468131 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.468031 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:35:51.470509 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.470486 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"image-registry-7568cffd97-p78mw\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:35:51.470686 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.470665 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33485243-8457-4ab8-83a2-3ad6cc1d98f9-metrics-tls\") pod \"dns-default-wvj7q\" (UID: \"33485243-8457-4ab8-83a2-3ad6cc1d98f9\") " pod="openshift-dns/dns-default-wvj7q" Jul 02 21:35:51.470869 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.470848 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/701b2907-f252-40d2-a8f9-b22923946c39-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-w7bjn\" (UID: \"701b2907-f252-40d2-a8f9-b22923946c39\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:35:51.660651 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.660621 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-knz6t\"" Jul 02 21:35:51.660651 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.660622 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-b4gbg\"" Jul 02 21:35:51.660918 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.660651 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-24f84\"" Jul 02 21:35:51.667639 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.667609 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:35:51.667639 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.667636 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvj7q" Jul 02 21:35:51.667788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.667666 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" Jul 02 21:35:51.830244 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:51.830213 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wvj7q"] Jul 02 21:35:51.833113 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:51.833088 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33485243_8457_4ab8_83a2_3ad6cc1d98f9.slice/crio-f3e916e5cd1b5ab1d3ff937fb370657b94fc9119198c83e99424a55ebb933e2c WatchSource:0}: Error finding container f3e916e5cd1b5ab1d3ff937fb370657b94fc9119198c83e99424a55ebb933e2c: Status 404 returned error can't find the container with id f3e916e5cd1b5ab1d3ff937fb370657b94fc9119198c83e99424a55ebb933e2c Jul 02 21:35:52.051759 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:52.051617 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn"] Jul 02 21:35:52.054558 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:52.054521 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701b2907_f252_40d2_a8f9_b22923946c39.slice/crio-aec1da40f9ef27de04db0bd7b40c7a64224bdbb02b4fd871cd6e48bbaaad0032 WatchSource:0}: Error finding container aec1da40f9ef27de04db0bd7b40c7a64224bdbb02b4fd871cd6e48bbaaad0032: Status 404 returned error can't find the container with id aec1da40f9ef27de04db0bd7b40c7a64224bdbb02b4fd871cd6e48bbaaad0032 Jul 02 21:35:52.054966 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:52.054943 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7568cffd97-p78mw"] Jul 02 21:35:52.058137 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:52.058118 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde89456c_8281_4f09_84b8_6b2102028ff0.slice/crio-e1e9e0ba7d407588b72c148a1e3e1e5e5929ee643f5b5277d7e2d2d199b6b344 WatchSource:0}: Error finding container e1e9e0ba7d407588b72c148a1e3e1e5e5929ee643f5b5277d7e2d2d199b6b344: Status 404 returned error can't find the container with id e1e9e0ba7d407588b72c148a1e3e1e5e5929ee643f5b5277d7e2d2d199b6b344 Jul 02 21:35:52.076491 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:52.076460 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" event={"ID":"2b04bf6c-d4fd-4686-88ce-7c896a9bf1e9","Type":"ContainerStarted","Data":"a2d118021f3d0f651bf58aad895668a6c65f53adbe9d56988d5a3f08c1ccefb1"} Jul 02 21:35:52.076963 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:52.076926 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" Jul 02 21:35:52.077950 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:52.077925 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" event={"ID":"701b2907-f252-40d2-a8f9-b22923946c39","Type":"ContainerStarted","Data":"aec1da40f9ef27de04db0bd7b40c7a64224bdbb02b4fd871cd6e48bbaaad0032"} Jul 02 21:35:52.079094 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:52.079068 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" event={"ID":"de89456c-8281-4f09-84b8-6b2102028ff0","Type":"ContainerStarted","Data":"e1e9e0ba7d407588b72c148a1e3e1e5e5929ee643f5b5277d7e2d2d199b6b344"} Jul 02 21:35:52.080208 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:52.080180 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvj7q" event={"ID":"33485243-8457-4ab8-83a2-3ad6cc1d98f9","Type":"ContainerStarted","Data":"f3e916e5cd1b5ab1d3ff937fb370657b94fc9119198c83e99424a55ebb933e2c"} Jul 02 21:35:52.081456 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:52.081439 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" Jul 02 21:35:52.092504 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:52.092451 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-28k47" podStartSLOduration=1.918321583 podStartE2EDuration="3.092430309s" totalImagesPullingTime="1.174108726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:35:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:52.091561136 +0000 UTC m=+286.541399760" watchObservedRunningTime="2026-07-02 21:35:52.092430309 +0000 UTC m=+286.542268933" Jul 02 21:35:53.005681 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.005647 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x"] Jul 02 21:35:53.009041 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.009018 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.012903 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.012878 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 02 21:35:53.013072 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.012947 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 21:35:53.013072 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.012878 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-6wtmp\"" Jul 02 21:35:53.013204 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.013135 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 02 21:35:53.019356 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.018617 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x"] Jul 02 21:35:53.083340 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.083309 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.083773 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.083397 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.083773 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.083448 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.083773 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.083489 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz2sb\" (UniqueName: \"kubernetes.io/projected/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-kube-api-access-mz2sb\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.085770 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.085697 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" event={"ID":"de89456c-8281-4f09-84b8-6b2102028ff0","Type":"ContainerStarted","Data":"aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466"} Jul 02 21:35:53.085933 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.085915 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:35:53.106296 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.106239 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" podStartSLOduration=287.10621751 podStartE2EDuration="4m47.10621751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:53.104676475 +0000 UTC m=+287.554515102" watchObservedRunningTime="2026-07-02 21:35:53.10621751 +0000 UTC m=+287.556056135" Jul 02 21:35:53.184622 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.184580 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.184810 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.184643 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.184810 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.184671 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz2sb\" (UniqueName: \"kubernetes.io/projected/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-kube-api-access-mz2sb\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.184810 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:53.184783 2547 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 02 21:35:53.184996 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:53.184853 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-prometheus-operator-tls podName:5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9 nodeName:}" failed. No retries permitted until 2026-07-02 21:35:53.684831858 +0000 UTC m=+288.134670479 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-cxh5x" (UID: "5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9") : secret "prometheus-operator-tls" not found Jul 02 21:35:53.185256 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.185229 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.185441 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.185421 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.188070 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.188042 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.193830 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.193805 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz2sb\" (UniqueName: \"kubernetes.io/projected/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-kube-api-access-mz2sb\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.690683 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.690638 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.693451 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.693419 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-cxh5x\" (UID: \"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:53.922027 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:53.922003 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" Jul 02 21:35:54.062677 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:54.062649 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x"] Jul 02 21:35:54.065191 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:54.065165 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b3d15c9_23cf_4bb1_8a67_d23f31e6cdf9.slice/crio-90323e7a195b6e2ef0b8d52e0bf734c6bd52a145962eafda8e01e2d4627fdca0 WatchSource:0}: Error finding container 90323e7a195b6e2ef0b8d52e0bf734c6bd52a145962eafda8e01e2d4627fdca0: Status 404 returned error can't find the container with id 90323e7a195b6e2ef0b8d52e0bf734c6bd52a145962eafda8e01e2d4627fdca0 Jul 02 21:35:54.090960 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:54.090929 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" event={"ID":"701b2907-f252-40d2-a8f9-b22923946c39","Type":"ContainerStarted","Data":"4f0ca19a6878de1cebfa97bb9ec2abd5c6c9efd72ac6a59efec555cc0ed0e5dc"} Jul 02 21:35:54.092091 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:54.092069 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" event={"ID":"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9","Type":"ContainerStarted","Data":"90323e7a195b6e2ef0b8d52e0bf734c6bd52a145962eafda8e01e2d4627fdca0"} Jul 02 21:35:54.098042 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:54.098016 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvj7q" event={"ID":"33485243-8457-4ab8-83a2-3ad6cc1d98f9","Type":"ContainerStarted","Data":"9b08ade46d7e41e6de9ca04c3590a9b2019b0e062a6105e5b0a5bfc1d3dd94ac"} Jul 02 21:35:54.098150 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:54.098094 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvj7q" event={"ID":"33485243-8457-4ab8-83a2-3ad6cc1d98f9","Type":"ContainerStarted","Data":"0845c3cabdc657afcd80c5ce6b1d589b79837e033e9bd3bfd7ee4da8073e0813"} Jul 02 21:35:54.107564 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:54.107530 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-w7bjn" podStartSLOduration=276.331368294 podStartE2EDuration="4m38.107519263s" totalImagesPullingTime="1.776150969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:31:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:54.107004685 +0000 UTC m=+288.556843307" watchObservedRunningTime="2026-07-02 21:35:54.107519263 +0000 UTC m=+288.557357905" Jul 02 21:35:54.124304 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:54.124259 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wvj7q" podStartSLOduration=251.121576269 podStartE2EDuration="4m13.124240584s" totalImagesPullingTime="2.002664315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:54.124228621 +0000 UTC m=+288.574067245" watchObservedRunningTime="2026-07-02 21:35:54.124240584 +0000 UTC m=+288.574079210" Jul 02 21:35:55.102459 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:55.102419 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wvj7q" Jul 02 21:35:56.106700 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:56.106667 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" event={"ID":"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9","Type":"ContainerStarted","Data":"5dd73a2c15103030e2b7e594f64a24961ab1e6eecbdea9874402ffcdb21bc1c2"} Jul 02 21:35:56.107145 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:56.106721 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" event={"ID":"5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9","Type":"ContainerStarted","Data":"6053612c26e8a4d4336e3aa8c0c7f2fa89aa0755a05fec84b6ac22f70b5d9165"} Jul 02 21:35:56.125165 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:56.125112 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-cxh5x" podStartSLOduration=2.832739754 podStartE2EDuration="4.1250944s" totalImagesPullingTime="1.292354646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:35:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:35:56.124628325 +0000 UTC m=+290.574466948" watchObservedRunningTime="2026-07-02 21:35:56.1250944 +0000 UTC m=+290.574933025" Jul 02 21:35:58.348873 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.348838 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h"] Jul 02 21:35:58.355725 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.355696 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mjz5t"] Jul 02 21:35:58.355847 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.355818 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.358850 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.358826 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 02 21:35:58.358992 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.358842 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 02 21:35:58.359253 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.359240 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 02 21:35:58.359364 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.359346 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mvbx2\"" Jul 02 21:35:58.359423 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.359368 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.362331 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.362306 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 21:35:58.362495 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.362397 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vcgfq\"" Jul 02 21:35:58.362672 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.362651 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 21:35:58.363090 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.363073 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 21:35:58.363355 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.363336 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h"] Jul 02 21:35:58.429576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429549 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-accelerators-collector-config\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.429576 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429579 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v2td\" (UniqueName: \"kubernetes.io/projected/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-api-access-8v2td\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.429764 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429600 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-textfile\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.429764 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429614 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5zfd\" (UniqueName: \"kubernetes.io/projected/ff06f541-081d-4943-89c3-00a786913e29-kube-api-access-t5zfd\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.429764 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429648 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff06f541-081d-4943-89c3-00a786913e29-metrics-client-ca\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.429764 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429726 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff06f541-081d-4943-89c3-00a786913e29-sys\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.429764 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429755 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-wtmp\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.429911 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429779 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.429911 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429805 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/456f8a1d-f937-458a-8063-5ebc0ee7da5b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.429911 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429829 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ff06f541-081d-4943-89c3-00a786913e29-root\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.429911 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429847 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-tls\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.429911 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429872 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.429911 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429891 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/456f8a1d-f937-458a-8063-5ebc0ee7da5b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.429911 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429906 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.430111 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.429940 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.530368 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530336 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.530579 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530405 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-accelerators-collector-config\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530579 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530434 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v2td\" (UniqueName: \"kubernetes.io/projected/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-api-access-8v2td\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.530579 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530457 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-textfile\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530579 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530472 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5zfd\" (UniqueName: \"kubernetes.io/projected/ff06f541-081d-4943-89c3-00a786913e29-kube-api-access-t5zfd\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530579 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530490 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff06f541-081d-4943-89c3-00a786913e29-metrics-client-ca\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530579 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530518 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff06f541-081d-4943-89c3-00a786913e29-sys\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530579 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530546 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-wtmp\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530590 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530613 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ff06f541-081d-4943-89c3-00a786913e29-sys\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530617 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/456f8a1d-f937-458a-8063-5ebc0ee7da5b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530736 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ff06f541-081d-4943-89c3-00a786913e29-root\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530742 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-wtmp\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530769 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-tls\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530814 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530834 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-textfile\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530845 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/456f8a1d-f937-458a-8063-5ebc0ee7da5b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.530956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.530883 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.531539 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:58.530995 2547 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 02 21:35:58.531539 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:35:58.531050 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-tls podName:456f8a1d-f937-458a-8063-5ebc0ee7da5b nodeName:}" failed. No retries permitted until 2026-07-02 21:35:59.031032301 +0000 UTC m=+293.480870916 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-f786h" (UID: "456f8a1d-f937-458a-8063-5ebc0ee7da5b") : secret "kube-state-metrics-tls" not found Jul 02 21:35:58.531539 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.531052 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/456f8a1d-f937-458a-8063-5ebc0ee7da5b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.531539 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.531250 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff06f541-081d-4943-89c3-00a786913e29-metrics-client-ca\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.531539 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.531308 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-accelerators-collector-config\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.531539 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.531322 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ff06f541-081d-4943-89c3-00a786913e29-root\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.531539 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.531323 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.531780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.531561 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/456f8a1d-f937-458a-8063-5ebc0ee7da5b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.532982 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.532962 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.533278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.533262 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-tls\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.533338 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.533325 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ff06f541-081d-4943-89c3-00a786913e29-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.541285 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.541260 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5zfd\" (UniqueName: \"kubernetes.io/projected/ff06f541-081d-4943-89c3-00a786913e29-kube-api-access-t5zfd\") pod \"node-exporter-mjz5t\" (UID: \"ff06f541-081d-4943-89c3-00a786913e29\") " pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.541643 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.541626 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v2td\" (UniqueName: \"kubernetes.io/projected/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-api-access-8v2td\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:58.672534 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:58.672455 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mjz5t" Jul 02 21:35:58.684089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:58.684051 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff06f541_081d_4943_89c3_00a786913e29.slice/crio-3d6fc85859092c9111ef6b03c704d580e62974180100940d03b2c349eb262d0e WatchSource:0}: Error finding container 3d6fc85859092c9111ef6b03c704d580e62974180100940d03b2c349eb262d0e: Status 404 returned error can't find the container with id 3d6fc85859092c9111ef6b03c704d580e62974180100940d03b2c349eb262d0e Jul 02 21:35:59.036730 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:59.036642 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:59.038986 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:59.038962 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/456f8a1d-f937-458a-8063-5ebc0ee7da5b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-f786h\" (UID: \"456f8a1d-f937-458a-8063-5ebc0ee7da5b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:59.117202 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:59.117167 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjz5t" event={"ID":"ff06f541-081d-4943-89c3-00a786913e29","Type":"ContainerStarted","Data":"3d6fc85859092c9111ef6b03c704d580e62974180100940d03b2c349eb262d0e"} Jul 02 21:35:59.267958 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:59.267921 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" Jul 02 21:35:59.438757 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:35:59.438590 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h"] Jul 02 21:35:59.632711 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:35:59.632640 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod456f8a1d_f937_458a_8063_5ebc0ee7da5b.slice/crio-13e16643249e2ead11f3bcad8268739ceed3cbd109a805ce7616dec48badf265 WatchSource:0}: Error finding container 13e16643249e2ead11f3bcad8268739ceed3cbd109a805ce7616dec48badf265: Status 404 returned error can't find the container with id 13e16643249e2ead11f3bcad8268739ceed3cbd109a805ce7616dec48badf265 Jul 02 21:36:00.122364 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.122327 2547 generic.go:356] "Generic (PLEG): container finished" podID="ff06f541-081d-4943-89c3-00a786913e29" containerID="c9304111b9485ebaf09d5ed523e5bd8373843867b299ee346f5da452919a9d4c" exitCode=0 Jul 02 21:36:00.122565 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.122366 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjz5t" event={"ID":"ff06f541-081d-4943-89c3-00a786913e29","Type":"ContainerDied","Data":"c9304111b9485ebaf09d5ed523e5bd8373843867b299ee346f5da452919a9d4c"} Jul 02 21:36:00.123598 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.123570 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" event={"ID":"456f8a1d-f937-458a-8063-5ebc0ee7da5b","Type":"ContainerStarted","Data":"13e16643249e2ead11f3bcad8268739ceed3cbd109a805ce7616dec48badf265"} Jul 02 21:36:00.327641 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.325722 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-f5854f8c9-72pq4"] Jul 02 21:36:00.330131 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.330101 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.333081 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.333045 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 02 21:36:00.333230 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.333097 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 02 21:36:00.333230 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.333105 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 02 21:36:00.333230 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.333128 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-eb2g02fd18379\"" Jul 02 21:36:00.333230 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.333175 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 02 21:36:00.333230 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.333045 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 02 21:36:00.333230 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.333097 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zpjng\"" Jul 02 21:36:00.342176 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.342016 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f5854f8c9-72pq4"] Jul 02 21:36:00.448436 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.448355 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-tls\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.448436 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.448401 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwwng\" (UniqueName: \"kubernetes.io/projected/29f1e4dd-119c-4a61-aaf1-8736217793fd-kube-api-access-xwwng\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.448436 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.448427 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.448916 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.448507 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29f1e4dd-119c-4a61-aaf1-8736217793fd-metrics-client-ca\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.448916 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.448572 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.448916 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.448596 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.448916 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.448632 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-grpc-tls\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.448916 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.448652 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.549893 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.549853 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.550086 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.549924 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-tls\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.550086 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.549948 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwwng\" (UniqueName: \"kubernetes.io/projected/29f1e4dd-119c-4a61-aaf1-8736217793fd-kube-api-access-xwwng\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.550086 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.549981 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.550086 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.550014 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29f1e4dd-119c-4a61-aaf1-8736217793fd-metrics-client-ca\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.550293 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.550255 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.550355 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.550295 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.550450 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.550426 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-grpc-tls\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.550881 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.550830 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/29f1e4dd-119c-4a61-aaf1-8736217793fd-metrics-client-ca\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.553011 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.552965 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.553478 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.553313 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-tls\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.553478 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.553366 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.553634 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.553574 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.553702 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.553685 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.554659 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.554637 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/29f1e4dd-119c-4a61-aaf1-8736217793fd-secret-grpc-tls\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.558289 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.558267 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwwng\" (UniqueName: \"kubernetes.io/projected/29f1e4dd-119c-4a61-aaf1-8736217793fd-kube-api-access-xwwng\") pod \"thanos-querier-f5854f8c9-72pq4\" (UID: \"29f1e4dd-119c-4a61-aaf1-8736217793fd\") " pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.642899 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.642859 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:00.793853 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:00.793449 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f5854f8c9-72pq4"] Jul 02 21:36:00.998013 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:36:00.997950 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f1e4dd_119c_4a61_aaf1_8736217793fd.slice/crio-8f3ed4f5172238c42f70921cd7e0244a39fbf7a77f01f1b0a247fb13224cfba7 WatchSource:0}: Error finding container 8f3ed4f5172238c42f70921cd7e0244a39fbf7a77f01f1b0a247fb13224cfba7: Status 404 returned error can't find the container with id 8f3ed4f5172238c42f70921cd7e0244a39fbf7a77f01f1b0a247fb13224cfba7 Jul 02 21:36:01.129213 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:01.129173 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" event={"ID":"29f1e4dd-119c-4a61-aaf1-8736217793fd","Type":"ContainerStarted","Data":"8f3ed4f5172238c42f70921cd7e0244a39fbf7a77f01f1b0a247fb13224cfba7"} Jul 02 21:36:01.131663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:01.131632 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjz5t" event={"ID":"ff06f541-081d-4943-89c3-00a786913e29","Type":"ContainerStarted","Data":"2826aa990361a2b266944d0926ee19e7d1b171a72b393101aa0f33df8ad06898"} Jul 02 21:36:01.131784 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:01.131696 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjz5t" event={"ID":"ff06f541-081d-4943-89c3-00a786913e29","Type":"ContainerStarted","Data":"1b1a8c9916f0f319461f5ac3532c9df067256b07e2ae4bdcc2783ddaa22041df"} Jul 02 21:36:01.133235 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:01.133208 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" event={"ID":"456f8a1d-f937-458a-8063-5ebc0ee7da5b","Type":"ContainerStarted","Data":"a76328b96ee508213039b8ce642b57fc2fb79472412b01982a8fd2e0c310e104"} Jul 02 21:36:01.152205 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:01.152130 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mjz5t" podStartSLOduration=2.099978885 podStartE2EDuration="3.15210935s" totalImagesPullingTime="991.542021ms" totalInitContainerRuntime="60.588444ms" isStatefulPod=true podCreationTimestamp="2026-07-02 21:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:01.150756546 +0000 UTC m=+295.600595173" watchObservedRunningTime="2026-07-02 21:36:01.15210935 +0000 UTC m=+295.601947976" Jul 02 21:36:02.003030 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:36:02.002980 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-95lhp" podUID="f4f1ee0d-0554-4134-b502-08c8aefca7ec" Jul 02 21:36:02.139541 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:02.139501 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" event={"ID":"456f8a1d-f937-458a-8063-5ebc0ee7da5b","Type":"ContainerStarted","Data":"373b1807bbd319b09f4d8e9bd330cb0940a25e42a901d1ec4dd1282a46c081a4"} Jul 02 21:36:02.139723 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:02.139569 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" event={"ID":"456f8a1d-f937-458a-8063-5ebc0ee7da5b","Type":"ContainerStarted","Data":"cd4798637ffeba2109f043f238dc1a8fe94d89f727124f78f71888329b194976"} Jul 02 21:36:02.160235 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:02.160179 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-f786h" podStartSLOduration=2.753213985 podStartE2EDuration="4.160158824s" totalImagesPullingTime="1.406944839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:02.158142713 +0000 UTC m=+296.607981338" watchObservedRunningTime="2026-07-02 21:36:02.160158824 +0000 UTC m=+296.609997447" Jul 02 21:36:04.003517 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:04.003476 2547 scope.go:117] "RemoveContainer" containerID="9cbafff3dda2c6ebef5faf08fb1f40d45231dfe1491eb8360029a1368dede749" Jul 02 21:36:04.150071 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:04.150040 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:36:04.150249 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:04.150188 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" event={"ID":"6e011139-283a-47f3-8490-1011cd239396","Type":"ContainerStarted","Data":"6cfe29fc890898b330fa5abe09e943c54dbebf411e5abd88b5a41dd177066c1f"} Jul 02 21:36:04.151148 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:04.151128 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:36:04.153721 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:04.153601 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" event={"ID":"29f1e4dd-119c-4a61-aaf1-8736217793fd","Type":"ContainerStarted","Data":"a3230b3e907f462095ea7f291eae83e341eac63984c85a11a28da4458715b8fe"} Jul 02 21:36:04.153721 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:04.153668 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" event={"ID":"29f1e4dd-119c-4a61-aaf1-8736217793fd","Type":"ContainerStarted","Data":"a6d3c0a0a4747251f554e6fac465f2b25fc76348a21a9a5f045dbfb05bf57065"} Jul 02 21:36:04.153721 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:04.153710 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" event={"ID":"29f1e4dd-119c-4a61-aaf1-8736217793fd","Type":"ContainerStarted","Data":"b416f3d26f2c067466acb76899a746e771a4193c3f3213337b7b52a3663a4ca9"} Jul 02 21:36:04.168002 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:04.167945 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" podStartSLOduration=43.089804067 podStartE2EDuration="45.16792367s" totalImagesPullingTime="2.078119603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:04.166492477 +0000 UTC m=+298.616331101" watchObservedRunningTime="2026-07-02 21:36:04.16792367 +0000 UTC m=+298.617762295" Jul 02 21:36:05.108758 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.108723 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wvj7q" Jul 02 21:36:05.150747 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.150703 2547 patch_prober.go:28] interesting pod/console-operator-68b757fb67-j6j5h container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.11:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 21:36:05.150907 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.150779 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" podUID="6e011139-283a-47f3-8490-1011cd239396" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.11:8443/readyz\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 02 21:36:05.161145 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.161110 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" event={"ID":"29f1e4dd-119c-4a61-aaf1-8736217793fd","Type":"ContainerStarted","Data":"ae1ddd5927601635a99e82de5c72529cda19b5e83e6050d482c430964a2daf95"} Jul 02 21:36:05.161316 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.161196 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" event={"ID":"29f1e4dd-119c-4a61-aaf1-8736217793fd","Type":"ContainerStarted","Data":"a40db6d726b134d0b8ec8766698073c64a66233d535c16f34b031201907d89af"} Jul 02 21:36:05.161316 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.161243 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" event={"ID":"29f1e4dd-119c-4a61-aaf1-8736217793fd","Type":"ContainerStarted","Data":"4c37f6cdd087edc439afb6671cd903f64c070d5d741ce6f3f33f63d73ce78559"} Jul 02 21:36:05.185407 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.184119 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" podStartSLOduration=1.6902823869999999 podStartE2EDuration="5.184099175s" totalImagesPullingTime="3.493816788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:05.18111401 +0000 UTC m=+299.630952636" watchObservedRunningTime="2026-07-02 21:36:05.184099175 +0000 UTC m=+299.633937801" Jul 02 21:36:05.504181 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.504107 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-j6j5h" Jul 02 21:36:05.664070 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.664036 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-ptjll"] Jul 02 21:36:05.667295 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.667278 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-ptjll" Jul 02 21:36:05.669978 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.669959 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 21:36:05.670044 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.669959 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 21:36:05.670107 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.669956 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-vs6tl\"" Jul 02 21:36:05.675958 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.675938 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-ptjll"] Jul 02 21:36:05.802893 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.802864 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbmsf\" (UniqueName: \"kubernetes.io/projected/e65b4301-e5c4-4b64-b629-96dd9153001e-kube-api-access-mbmsf\") pod \"downloads-685bfb5f64-ptjll\" (UID: \"e65b4301-e5c4-4b64-b629-96dd9153001e\") " pod="openshift-console/downloads-685bfb5f64-ptjll" Jul 02 21:36:05.866571 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.866547 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:36:05.867458 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.867435 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:36:05.876857 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.876834 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:36:05.878324 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.878300 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:36:05.879921 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.879905 2547 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 21:36:05.903893 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.903870 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbmsf\" (UniqueName: \"kubernetes.io/projected/e65b4301-e5c4-4b64-b629-96dd9153001e-kube-api-access-mbmsf\") pod \"downloads-685bfb5f64-ptjll\" (UID: \"e65b4301-e5c4-4b64-b629-96dd9153001e\") " pod="openshift-console/downloads-685bfb5f64-ptjll" Jul 02 21:36:05.913772 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.913754 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 21:36:05.923930 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.923908 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 21:36:05.934294 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.934269 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbmsf\" (UniqueName: \"kubernetes.io/projected/e65b4301-e5c4-4b64-b629-96dd9153001e-kube-api-access-mbmsf\") pod \"downloads-685bfb5f64-ptjll\" (UID: \"e65b4301-e5c4-4b64-b629-96dd9153001e\") " pod="openshift-console/downloads-685bfb5f64-ptjll" Jul 02 21:36:05.978697 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.978664 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-vs6tl\"" Jul 02 21:36:05.986454 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:05.986433 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-ptjll" Jul 02 21:36:06.116581 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:06.116537 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-ptjll"] Jul 02 21:36:06.120150 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:36:06.120120 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode65b4301_e5c4_4b64_b629_96dd9153001e.slice/crio-bbc5547659d94c7fb762d8759c5decb1ac752886e3cb2caa5549bd210161fbf2 WatchSource:0}: Error finding container bbc5547659d94c7fb762d8759c5decb1ac752886e3cb2caa5549bd210161fbf2: Status 404 returned error can't find the container with id bbc5547659d94c7fb762d8759c5decb1ac752886e3cb2caa5549bd210161fbf2 Jul 02 21:36:06.165560 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:06.165527 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-ptjll" event={"ID":"e65b4301-e5c4-4b64-b629-96dd9153001e","Type":"ContainerStarted","Data":"bbc5547659d94c7fb762d8759c5decb1ac752886e3cb2caa5549bd210161fbf2"} Jul 02 21:36:06.165960 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:06.165944 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:11.198255 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.198205 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68b4d96fb5-95wvc"] Jul 02 21:36:11.202297 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.202271 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.205872 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.205199 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 21:36:11.206223 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.206205 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 21:36:11.206508 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.206470 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 21:36:11.206631 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.206470 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 21:36:11.206631 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.206618 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-x5d97\"" Jul 02 21:36:11.206759 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.206480 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 21:36:11.215813 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.215788 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68b4d96fb5-95wvc"] Jul 02 21:36:11.355348 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.355311 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-oauth-serving-cert\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.355537 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.355366 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-config\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.355537 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.355494 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-serving-cert\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.355656 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.355547 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp4r7\" (UniqueName: \"kubernetes.io/projected/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-kube-api-access-xp4r7\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.355656 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.355590 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-service-ca\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.355656 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.355623 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-oauth-config\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.387472 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.387442 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7568cffd97-p78mw"] Jul 02 21:36:11.393929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.393886 2547 patch_prober.go:28] interesting pod/image-registry-7568cffd97-p78mw container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 21:36:11.393929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.393946 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" podUID="de89456c-8281-4f09-84b8-6b2102028ff0" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 21:36:11.456696 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.456609 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-oauth-serving-cert\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.456696 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.456656 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-config\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.456896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.456716 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-serving-cert\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.456896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.456757 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp4r7\" (UniqueName: \"kubernetes.io/projected/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-kube-api-access-xp4r7\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.457295 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.457138 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-service-ca\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.457295 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.457207 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-oauth-config\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.457523 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.457492 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-oauth-serving-cert\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.457523 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.457509 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-config\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.457781 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.457754 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-service-ca\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.459919 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.459895 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-serving-cert\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.460007 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.459953 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-oauth-config\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.465336 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.465317 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp4r7\" (UniqueName: \"kubernetes.io/projected/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-kube-api-access-xp4r7\") pod \"console-68b4d96fb5-95wvc\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.515570 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.515508 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:11.670178 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.669850 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68b4d96fb5-95wvc"] Jul 02 21:36:11.673034 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:36:11.673003 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9488f3c6_93b1_4fa5_9ee7_503cb61cde26.slice/crio-eafca18a84bb34be31118b864c5241f882f73faf602c23251818b7a22a0c30b7 WatchSource:0}: Error finding container eafca18a84bb34be31118b864c5241f882f73faf602c23251818b7a22a0c30b7: Status 404 returned error can't find the container with id eafca18a84bb34be31118b864c5241f882f73faf602c23251818b7a22a0c30b7 Jul 02 21:36:11.675812 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:11.675789 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:36:12.178561 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:12.178531 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-f5854f8c9-72pq4" Jul 02 21:36:12.196885 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:12.196848 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68b4d96fb5-95wvc" event={"ID":"9488f3c6-93b1-4fa5-9ee7-503cb61cde26","Type":"ContainerStarted","Data":"eafca18a84bb34be31118b864c5241f882f73faf602c23251818b7a22a0c30b7"} Jul 02 21:36:13.002271 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:13.002239 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:36:16.216822 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:16.216787 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68b4d96fb5-95wvc" event={"ID":"9488f3c6-93b1-4fa5-9ee7-503cb61cde26","Type":"ContainerStarted","Data":"ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d"} Jul 02 21:36:16.236132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:16.236077 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68b4d96fb5-95wvc" podStartSLOduration=1.497975691 podStartE2EDuration="5.236057179s" totalImagesPullingTime="3.738081488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:16.233587421 +0000 UTC m=+310.683426045" watchObservedRunningTime="2026-07-02 21:36:16.236057179 +0000 UTC m=+310.685895787" Jul 02 21:36:20.780890 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.780854 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85b755d789-ds4bz"] Jul 02 21:36:20.793539 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.793507 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:20.794295 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.794269 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b755d789-ds4bz"] Jul 02 21:36:20.801635 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.801609 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 21:36:20.941674 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.941638 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-service-ca\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:20.941955 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.941729 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-oauth-config\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:20.941955 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.941765 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-trusted-ca-bundle\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:20.941955 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.941884 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-config\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:20.942121 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.941954 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-oauth-serving-cert\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:20.942121 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.941999 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-serving-cert\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:20.942121 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:20.942034 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7mwj\" (UniqueName: \"kubernetes.io/projected/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-kube-api-access-c7mwj\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.043123 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.043037 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-oauth-config\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.043123 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.043080 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-trusted-ca-bundle\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.043123 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.043111 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-config\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.043367 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.043144 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-oauth-serving-cert\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.043367 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.043170 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-serving-cert\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.043367 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.043189 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7mwj\" (UniqueName: \"kubernetes.io/projected/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-kube-api-access-c7mwj\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.043367 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.043239 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-service-ca\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.044032 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.043974 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-oauth-serving-cert\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.044157 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.044038 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-service-ca\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.044157 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.044057 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-trusted-ca-bundle\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.044258 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.044203 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-config\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.045839 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.045815 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-oauth-config\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.046184 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.046158 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-serving-cert\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.051671 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.051648 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7mwj\" (UniqueName: \"kubernetes.io/projected/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-kube-api-access-c7mwj\") pod \"console-85b755d789-ds4bz\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.105143 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.105104 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:21.394246 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.393562 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:36:21.516745 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.516695 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:21.517048 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.517024 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:21.522872 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:21.522849 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:22.250026 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:22.249999 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:23.208918 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:23.208897 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85b755d789-ds4bz"] Jul 02 21:36:23.211587 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:36:23.211558 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0e50f9_b9ed_490f_96f0_0c934b2bfd4f.slice/crio-a13e00b022f230bc1f305f9612662f853d3ecff25c8ed72e137dbc2fc38bf077 WatchSource:0}: Error finding container a13e00b022f230bc1f305f9612662f853d3ecff25c8ed72e137dbc2fc38bf077: Status 404 returned error can't find the container with id a13e00b022f230bc1f305f9612662f853d3ecff25c8ed72e137dbc2fc38bf077 Jul 02 21:36:23.251570 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:23.251545 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b755d789-ds4bz" event={"ID":"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f","Type":"ContainerStarted","Data":"a13e00b022f230bc1f305f9612662f853d3ecff25c8ed72e137dbc2fc38bf077"} Jul 02 21:36:24.259394 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:24.259305 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-ptjll" event={"ID":"e65b4301-e5c4-4b64-b629-96dd9153001e","Type":"ContainerStarted","Data":"c9a6b7be41eb6332ec3ab3473e32f2a33195de70a8f34f620becdef5e906d43b"} Jul 02 21:36:24.259879 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:24.259809 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-ptjll" Jul 02 21:36:24.261984 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:24.261946 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b755d789-ds4bz" event={"ID":"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f","Type":"ContainerStarted","Data":"41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a"} Jul 02 21:36:24.270726 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:24.270689 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-ptjll" Jul 02 21:36:24.283955 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:24.283894 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-ptjll" podStartSLOduration=2.241213366 podStartE2EDuration="19.28387193s" totalImagesPullingTime="17.042658564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:24.2835324 +0000 UTC m=+318.733371025" watchObservedRunningTime="2026-07-02 21:36:24.28387193 +0000 UTC m=+318.733710558" Jul 02 21:36:24.309925 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:24.309867 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85b755d789-ds4bz" podStartSLOduration=4.309851434 podStartE2EDuration="4.309851434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:36:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:36:24.30783179 +0000 UTC m=+318.757670415" watchObservedRunningTime="2026-07-02 21:36:24.309851434 +0000 UTC m=+318.759690123" Jul 02 21:36:31.105759 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:31.105714 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:31.106277 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:31.105786 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:31.111206 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:31.111174 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:31.300245 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:31.300217 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:36:31.352106 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:31.352070 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68b4d96fb5-95wvc"] Jul 02 21:36:36.415122 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.415075 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" podUID="de89456c-8281-4f09-84b8-6b2102028ff0" containerName="registry" containerID="cri-o://aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466" gracePeriod=30 Jul 02 21:36:36.703555 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.703520 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:36:36.787828 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.787790 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-installation-pull-secrets\") pod \"de89456c-8281-4f09-84b8-6b2102028ff0\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " Jul 02 21:36:36.787996 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.787865 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-bound-sa-token\") pod \"de89456c-8281-4f09-84b8-6b2102028ff0\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " Jul 02 21:36:36.787996 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.787897 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de89456c-8281-4f09-84b8-6b2102028ff0-ca-trust-extracted\") pod \"de89456c-8281-4f09-84b8-6b2102028ff0\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " Jul 02 21:36:36.787996 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.787920 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") pod \"de89456c-8281-4f09-84b8-6b2102028ff0\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " Jul 02 21:36:36.787996 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.787957 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-registry-certificates\") pod \"de89456c-8281-4f09-84b8-6b2102028ff0\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " Jul 02 21:36:36.787996 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.787986 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-trusted-ca\") pod \"de89456c-8281-4f09-84b8-6b2102028ff0\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " Jul 02 21:36:36.788242 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.788012 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-image-registry-private-configuration\") pod \"de89456c-8281-4f09-84b8-6b2102028ff0\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " Jul 02 21:36:36.788242 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.788041 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg9b2\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-kube-api-access-pg9b2\") pod \"de89456c-8281-4f09-84b8-6b2102028ff0\" (UID: \"de89456c-8281-4f09-84b8-6b2102028ff0\") " Jul 02 21:36:36.788515 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.788486 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "de89456c-8281-4f09-84b8-6b2102028ff0" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:36:36.788708 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.788677 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "de89456c-8281-4f09-84b8-6b2102028ff0" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:36:36.790342 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.790311 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "de89456c-8281-4f09-84b8-6b2102028ff0" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:36:36.790541 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.790515 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-kube-api-access-pg9b2" (OuterVolumeSpecName: "kube-api-access-pg9b2") pod "de89456c-8281-4f09-84b8-6b2102028ff0" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0"). InnerVolumeSpecName "kube-api-access-pg9b2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:36:36.790541 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.790526 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "de89456c-8281-4f09-84b8-6b2102028ff0" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:36:36.790687 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.790590 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "de89456c-8281-4f09-84b8-6b2102028ff0" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:36:36.790745 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.790729 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "de89456c-8281-4f09-84b8-6b2102028ff0" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:36:36.796691 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.796669 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de89456c-8281-4f09-84b8-6b2102028ff0-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "de89456c-8281-4f09-84b8-6b2102028ff0" (UID: "de89456c-8281-4f09-84b8-6b2102028ff0"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:36:36.889455 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.889419 2547 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-installation-pull-secrets\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:36.889455 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.889448 2547 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-bound-sa-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:36.889455 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.889460 2547 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de89456c-8281-4f09-84b8-6b2102028ff0-ca-trust-extracted\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:36.889672 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.889469 2547 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-registry-tls\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:36.889672 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.889478 2547 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-registry-certificates\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:36.889672 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.889486 2547 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de89456c-8281-4f09-84b8-6b2102028ff0-trusted-ca\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:36.889672 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.889495 2547 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de89456c-8281-4f09-84b8-6b2102028ff0-image-registry-private-configuration\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:36.889672 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:36.889504 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg9b2\" (UniqueName: \"kubernetes.io/projected/de89456c-8281-4f09-84b8-6b2102028ff0-kube-api-access-pg9b2\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:37.319733 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:37.319702 2547 generic.go:356] "Generic (PLEG): container finished" podID="de89456c-8281-4f09-84b8-6b2102028ff0" containerID="aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466" exitCode=0 Jul 02 21:36:37.319914 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:37.319743 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" Jul 02 21:36:37.319914 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:37.319796 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" event={"ID":"de89456c-8281-4f09-84b8-6b2102028ff0","Type":"ContainerDied","Data":"aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466"} Jul 02 21:36:37.319914 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:37.319846 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7568cffd97-p78mw" event={"ID":"de89456c-8281-4f09-84b8-6b2102028ff0","Type":"ContainerDied","Data":"e1e9e0ba7d407588b72c148a1e3e1e5e5929ee643f5b5277d7e2d2d199b6b344"} Jul 02 21:36:37.319914 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:37.319870 2547 scope.go:117] "RemoveContainer" containerID="aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466" Jul 02 21:36:37.329255 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:37.329240 2547 scope.go:117] "RemoveContainer" containerID="aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466" Jul 02 21:36:37.329539 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:36:37.329519 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466\": container with ID starting with aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466 not found: ID does not exist" containerID="aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466" Jul 02 21:36:37.329595 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:37.329547 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466"} err="failed to get container status \"aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466\": rpc error: code = NotFound desc = could not find container \"aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466\": container with ID starting with aa5757387df1943d2a35e644f55866e5996f0fc4f459bc2888595fe71cda1466 not found: ID does not exist" Jul 02 21:36:37.341042 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:37.341019 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7568cffd97-p78mw"] Jul 02 21:36:37.344512 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:37.344492 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7568cffd97-p78mw"] Jul 02 21:36:38.007746 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:38.007714 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de89456c-8281-4f09-84b8-6b2102028ff0" path="/var/lib/kubelet/pods/de89456c-8281-4f09-84b8-6b2102028ff0/volumes" Jul 02 21:36:56.378810 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.378741 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-68b4d96fb5-95wvc" podUID="9488f3c6-93b1-4fa5-9ee7-503cb61cde26" containerName="console" containerID="cri-o://ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d" gracePeriod=15 Jul 02 21:36:56.667174 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.667141 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:56.767569 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.767540 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-oauth-serving-cert\") pod \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " Jul 02 21:36:56.767569 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.767573 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-serving-cert\") pod \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " Jul 02 21:36:56.767831 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.767608 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-service-ca\") pod \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " Jul 02 21:36:56.767831 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.767638 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp4r7\" (UniqueName: \"kubernetes.io/projected/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-kube-api-access-xp4r7\") pod \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " Jul 02 21:36:56.767831 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.767684 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-oauth-config\") pod \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " Jul 02 21:36:56.767831 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.767712 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-config\") pod \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\" (UID: \"9488f3c6-93b1-4fa5-9ee7-503cb61cde26\") " Jul 02 21:36:56.768131 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.768097 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9488f3c6-93b1-4fa5-9ee7-503cb61cde26" (UID: "9488f3c6-93b1-4fa5-9ee7-503cb61cde26"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:36:56.768198 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.768108 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-service-ca" (OuterVolumeSpecName: "service-ca") pod "9488f3c6-93b1-4fa5-9ee7-503cb61cde26" (UID: "9488f3c6-93b1-4fa5-9ee7-503cb61cde26"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:36:56.768198 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.768165 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-config" (OuterVolumeSpecName: "console-config") pod "9488f3c6-93b1-4fa5-9ee7-503cb61cde26" (UID: "9488f3c6-93b1-4fa5-9ee7-503cb61cde26"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:36:56.769935 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.769905 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9488f3c6-93b1-4fa5-9ee7-503cb61cde26" (UID: "9488f3c6-93b1-4fa5-9ee7-503cb61cde26"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:36:56.770046 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.769931 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9488f3c6-93b1-4fa5-9ee7-503cb61cde26" (UID: "9488f3c6-93b1-4fa5-9ee7-503cb61cde26"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:36:56.770046 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.769948 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-kube-api-access-xp4r7" (OuterVolumeSpecName: "kube-api-access-xp4r7") pod "9488f3c6-93b1-4fa5-9ee7-503cb61cde26" (UID: "9488f3c6-93b1-4fa5-9ee7-503cb61cde26"). InnerVolumeSpecName "kube-api-access-xp4r7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:36:56.868671 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.868635 2547 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-oauth-serving-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:56.868671 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.868665 2547 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-serving-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:56.868671 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.868678 2547 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-service-ca\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:56.868908 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.868692 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp4r7\" (UniqueName: \"kubernetes.io/projected/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-kube-api-access-xp4r7\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:56.868908 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.868704 2547 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-oauth-config\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:56.868908 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:56.868716 2547 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9488f3c6-93b1-4fa5-9ee7-503cb61cde26-console-config\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:36:57.398761 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:57.398652 2547 generic.go:356] "Generic (PLEG): container finished" podID="9488f3c6-93b1-4fa5-9ee7-503cb61cde26" containerID="ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d" exitCode=0 Jul 02 21:36:57.398761 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:57.398709 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68b4d96fb5-95wvc" Jul 02 21:36:57.398761 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:57.398721 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68b4d96fb5-95wvc" event={"ID":"9488f3c6-93b1-4fa5-9ee7-503cb61cde26","Type":"ContainerDied","Data":"ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d"} Jul 02 21:36:57.398761 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:57.398763 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68b4d96fb5-95wvc" event={"ID":"9488f3c6-93b1-4fa5-9ee7-503cb61cde26","Type":"ContainerDied","Data":"eafca18a84bb34be31118b864c5241f882f73faf602c23251818b7a22a0c30b7"} Jul 02 21:36:57.399399 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:57.398782 2547 scope.go:117] "RemoveContainer" containerID="ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d" Jul 02 21:36:57.412522 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:57.412431 2547 scope.go:117] "RemoveContainer" containerID="ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d" Jul 02 21:36:57.412737 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:36:57.412710 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d\": container with ID starting with ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d not found: ID does not exist" containerID="ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d" Jul 02 21:36:57.412787 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:57.412749 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d"} err="failed to get container status \"ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d\": rpc error: code = NotFound desc = could not find container \"ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d\": container with ID starting with ab58baa4f24e20098e278f60e301ecbeb1fbcf32a5a21b28e1b6a1120445751d not found: ID does not exist" Jul 02 21:36:57.422509 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:57.422488 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68b4d96fb5-95wvc"] Jul 02 21:36:57.426728 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:57.426707 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68b4d96fb5-95wvc"] Jul 02 21:36:58.007038 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:36:58.007003 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9488f3c6-93b1-4fa5-9ee7-503cb61cde26" path="/var/lib/kubelet/pods/9488f3c6-93b1-4fa5-9ee7-503cb61cde26/volumes" Jul 02 21:37:18.957932 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:18.957889 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:37:18.960342 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:18.960315 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f4f1ee0d-0554-4134-b502-08c8aefca7ec-metrics-certs\") pod \"network-metrics-daemon-95lhp\" (UID: \"f4f1ee0d-0554-4134-b502-08c8aefca7ec\") " pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:37:19.005969 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:19.005940 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7x7kj\"" Jul 02 21:37:19.013989 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:19.013967 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-95lhp" Jul 02 21:37:19.142214 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:19.142162 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-95lhp"] Jul 02 21:37:19.144875 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:37:19.144844 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4f1ee0d_0554_4134_b502_08c8aefca7ec.slice/crio-53ad28b38dfa23b81d2856eec5a4c852713a7067ba24e209ca3d7c1c17eb3aa8 WatchSource:0}: Error finding container 53ad28b38dfa23b81d2856eec5a4c852713a7067ba24e209ca3d7c1c17eb3aa8: Status 404 returned error can't find the container with id 53ad28b38dfa23b81d2856eec5a4c852713a7067ba24e209ca3d7c1c17eb3aa8 Jul 02 21:37:19.491158 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:19.491117 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-95lhp" event={"ID":"f4f1ee0d-0554-4134-b502-08c8aefca7ec","Type":"ContainerStarted","Data":"53ad28b38dfa23b81d2856eec5a4c852713a7067ba24e209ca3d7c1c17eb3aa8"} Jul 02 21:37:20.496735 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:20.496640 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-95lhp" event={"ID":"f4f1ee0d-0554-4134-b502-08c8aefca7ec","Type":"ContainerStarted","Data":"0225c34ba19ba011d6f10f1c7f43d60ef7d0b102aba68e5fe690284dfffa8cba"} Jul 02 21:37:20.496735 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:20.496701 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-95lhp" event={"ID":"f4f1ee0d-0554-4134-b502-08c8aefca7ec","Type":"ContainerStarted","Data":"fbc34924891f7bf7142ff1a760f944d25b0aad6d8c87e8882f1e060793741d8c"} Jul 02 21:37:20.513628 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:20.513575 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-95lhp" podStartSLOduration=373.609584179 podStartE2EDuration="6m14.513561434s" totalImagesPullingTime="903.977255ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:20.511690984 +0000 UTC m=+374.961529609" watchObservedRunningTime="2026-07-02 21:37:20.513561434 +0000 UTC m=+374.963400057" Jul 02 21:37:22.640116 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.640079 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-c877d8d45-jrxrc"] Jul 02 21:37:22.641103 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.641078 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de89456c-8281-4f09-84b8-6b2102028ff0" containerName="registry" Jul 02 21:37:22.641103 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.641104 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="de89456c-8281-4f09-84b8-6b2102028ff0" containerName="registry" Jul 02 21:37:22.641298 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.641119 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9488f3c6-93b1-4fa5-9ee7-503cb61cde26" containerName="console" Jul 02 21:37:22.641298 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.641128 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="9488f3c6-93b1-4fa5-9ee7-503cb61cde26" containerName="console" Jul 02 21:37:22.641298 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.641205 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="9488f3c6-93b1-4fa5-9ee7-503cb61cde26" containerName="console" Jul 02 21:37:22.641298 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.641218 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="de89456c-8281-4f09-84b8-6b2102028ff0" containerName="registry" Jul 02 21:37:22.644851 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.644831 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.648918 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.648714 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 02 21:37:22.649425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.649074 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-v6b8k\"" Jul 02 21:37:22.649425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.649231 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 02 21:37:22.649425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.649393 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 02 21:37:22.649584 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.649479 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 02 21:37:22.649676 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.649660 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 02 21:37:22.653737 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.653712 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c877d8d45-jrxrc"] Jul 02 21:37:22.655536 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.655513 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 02 21:37:22.692157 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.692117 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738a153d-aa14-4fb0-a51c-ccb26a5114c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.692352 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.692192 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/738a153d-aa14-4fb0-a51c-ccb26a5114c0-metrics-client-ca\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.692352 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.692231 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l5lq\" (UniqueName: \"kubernetes.io/projected/738a153d-aa14-4fb0-a51c-ccb26a5114c0-kube-api-access-4l5lq\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.692510 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.692374 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738a153d-aa14-4fb0-a51c-ccb26a5114c0-serving-certs-ca-bundle\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.692510 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.692434 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.692510 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.692475 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-secret-telemeter-client\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.692613 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.692562 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-telemeter-client-tls\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.692647 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.692617 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-federate-client-tls\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.793948 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.793901 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738a153d-aa14-4fb0-a51c-ccb26a5114c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.794149 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.793982 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/738a153d-aa14-4fb0-a51c-ccb26a5114c0-metrics-client-ca\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.794149 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.794010 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l5lq\" (UniqueName: \"kubernetes.io/projected/738a153d-aa14-4fb0-a51c-ccb26a5114c0-kube-api-access-4l5lq\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.794149 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.794057 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738a153d-aa14-4fb0-a51c-ccb26a5114c0-serving-certs-ca-bundle\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.794149 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.794080 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.794149 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.794119 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-secret-telemeter-client\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.794414 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.794170 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-telemeter-client-tls\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.794414 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.794223 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-federate-client-tls\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.794812 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.794759 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/738a153d-aa14-4fb0-a51c-ccb26a5114c0-metrics-client-ca\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.795132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.795097 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738a153d-aa14-4fb0-a51c-ccb26a5114c0-telemeter-trusted-ca-bundle\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.795333 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.795309 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/738a153d-aa14-4fb0-a51c-ccb26a5114c0-serving-certs-ca-bundle\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.796882 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.796852 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.797023 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.796970 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-telemeter-client-tls\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.797144 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.797128 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-federate-client-tls\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.797524 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.797509 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/738a153d-aa14-4fb0-a51c-ccb26a5114c0-secret-telemeter-client\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.802095 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.802076 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l5lq\" (UniqueName: \"kubernetes.io/projected/738a153d-aa14-4fb0-a51c-ccb26a5114c0-kube-api-access-4l5lq\") pod \"telemeter-client-c877d8d45-jrxrc\" (UID: \"738a153d-aa14-4fb0-a51c-ccb26a5114c0\") " pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:22.959317 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:22.959229 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" Jul 02 21:37:23.116874 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:23.116845 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-c877d8d45-jrxrc"] Jul 02 21:37:23.121011 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:37:23.120255 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod738a153d_aa14_4fb0_a51c_ccb26a5114c0.slice/crio-fc65a7fb0cc9c0f418226f22a53f5f83d9fe52de49441323cab11761fbe7be09 WatchSource:0}: Error finding container fc65a7fb0cc9c0f418226f22a53f5f83d9fe52de49441323cab11761fbe7be09: Status 404 returned error can't find the container with id fc65a7fb0cc9c0f418226f22a53f5f83d9fe52de49441323cab11761fbe7be09 Jul 02 21:37:23.508773 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:23.508735 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" event={"ID":"738a153d-aa14-4fb0-a51c-ccb26a5114c0","Type":"ContainerStarted","Data":"fc65a7fb0cc9c0f418226f22a53f5f83d9fe52de49441323cab11761fbe7be09"} Jul 02 21:37:25.519485 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:25.519442 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" event={"ID":"738a153d-aa14-4fb0-a51c-ccb26a5114c0","Type":"ContainerStarted","Data":"3a52f49f95bd601d6c450e6a7d1658c01cc64f93609d661fecbf546e2ef544e5"} Jul 02 21:37:26.525555 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:26.525520 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" event={"ID":"738a153d-aa14-4fb0-a51c-ccb26a5114c0","Type":"ContainerStarted","Data":"0cd831c042cce85ffd4f73dba26a924b3791913e44172df80535c0269a8fab62"} Jul 02 21:37:26.525972 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:26.525577 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" event={"ID":"738a153d-aa14-4fb0-a51c-ccb26a5114c0","Type":"ContainerStarted","Data":"01401a19206b3aa44b5be18212ee1e5c616de0db7fa4549d327f765a3df210ee"} Jul 02 21:37:26.550849 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:26.550564 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-c877d8d45-jrxrc" podStartSLOduration=1.8247276719999999 podStartE2EDuration="4.550544392s" totalImagesPullingTime="2.72581672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:26.549765281 +0000 UTC m=+380.999603999" watchObservedRunningTime="2026-07-02 21:37:26.550544392 +0000 UTC m=+381.000383017" Jul 02 21:37:27.231821 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.231782 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85c9d966f6-dlwmw"] Jul 02 21:37:27.235391 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.235353 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.246895 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.246867 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85c9d966f6-dlwmw"] Jul 02 21:37:27.332788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.332754 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-oauth-serving-cert\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.332971 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.332801 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-oauth-config\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.332971 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.332820 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-console-config\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.332971 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.332903 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-trusted-ca-bundle\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.332971 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.332932 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8gc6\" (UniqueName: \"kubernetes.io/projected/e74653a9-f041-47a7-98f9-1f289b90b809-kube-api-access-z8gc6\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.333106 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.333015 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-service-ca\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.333106 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.333042 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-serving-cert\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.433414 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.433360 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-service-ca\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.433555 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.433422 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-serving-cert\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.433555 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.433482 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-oauth-serving-cert\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.433555 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.433523 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-oauth-config\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.433555 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.433550 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-console-config\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.433742 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.433585 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-trusted-ca-bundle\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.433742 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.433609 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8gc6\" (UniqueName: \"kubernetes.io/projected/e74653a9-f041-47a7-98f9-1f289b90b809-kube-api-access-z8gc6\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.434161 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.434130 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-service-ca\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.434280 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.434219 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-console-config\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.434280 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.434237 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-oauth-serving-cert\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.434484 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.434460 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-trusted-ca-bundle\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.435973 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.435943 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-oauth-config\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.436193 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.436173 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-serving-cert\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.443746 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.443724 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8gc6\" (UniqueName: \"kubernetes.io/projected/e74653a9-f041-47a7-98f9-1f289b90b809-kube-api-access-z8gc6\") pod \"console-85c9d966f6-dlwmw\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.547877 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.547851 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:27.675839 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:27.675816 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85c9d966f6-dlwmw"] Jul 02 21:37:27.678003 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:37:27.677972 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode74653a9_f041_47a7_98f9_1f289b90b809.slice/crio-6cafd0a9832bda81278ef539c041f407b43b3dbdda65bc21ef1729e400855156 WatchSource:0}: Error finding container 6cafd0a9832bda81278ef539c041f407b43b3dbdda65bc21ef1729e400855156: Status 404 returned error can't find the container with id 6cafd0a9832bda81278ef539c041f407b43b3dbdda65bc21ef1729e400855156 Jul 02 21:37:28.535903 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:28.535864 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c9d966f6-dlwmw" event={"ID":"e74653a9-f041-47a7-98f9-1f289b90b809","Type":"ContainerStarted","Data":"68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a"} Jul 02 21:37:28.536080 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:28.535920 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c9d966f6-dlwmw" event={"ID":"e74653a9-f041-47a7-98f9-1f289b90b809","Type":"ContainerStarted","Data":"6cafd0a9832bda81278ef539c041f407b43b3dbdda65bc21ef1729e400855156"} Jul 02 21:37:28.554651 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:28.554598 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85c9d966f6-dlwmw" podStartSLOduration=1.554582398 podStartE2EDuration="1.554582398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:37:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:37:28.554064575 +0000 UTC m=+383.003903202" watchObservedRunningTime="2026-07-02 21:37:28.554582398 +0000 UTC m=+383.004421022" Jul 02 21:37:37.548032 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:37.547992 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:37.548531 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:37.548059 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:37.552716 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:37.552696 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:37.576074 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:37.576046 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:37:37.623369 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:37:37.623338 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85b755d789-ds4bz"] Jul 02 21:38:02.649371 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:02.649306 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-85b755d789-ds4bz" podUID="3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" containerName="console" containerID="cri-o://41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a" gracePeriod=15 Jul 02 21:38:02.898938 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:02.898912 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:38:03.036290 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036209 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7mwj\" (UniqueName: \"kubernetes.io/projected/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-kube-api-access-c7mwj\") pod \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " Jul 02 21:38:03.036290 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036243 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-config\") pod \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " Jul 02 21:38:03.036290 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036260 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-oauth-serving-cert\") pod \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " Jul 02 21:38:03.036290 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036285 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-service-ca\") pod \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " Jul 02 21:38:03.036641 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036307 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-serving-cert\") pod \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " Jul 02 21:38:03.036641 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036374 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-oauth-config\") pod \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " Jul 02 21:38:03.036641 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036437 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-trusted-ca-bundle\") pod \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\" (UID: \"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f\") " Jul 02 21:38:03.036789 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036651 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-config" (OuterVolumeSpecName: "console-config") pod "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" (UID: "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:03.036789 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036710 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" (UID: "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:03.036789 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036718 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-service-ca" (OuterVolumeSpecName: "service-ca") pod "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" (UID: "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:03.036970 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.036941 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" (UID: "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:38:03.038433 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.038403 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-kube-api-access-c7mwj" (OuterVolumeSpecName: "kube-api-access-c7mwj") pod "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" (UID: "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f"). InnerVolumeSpecName "kube-api-access-c7mwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:38:03.038551 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.038499 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" (UID: "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:03.038551 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.038517 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" (UID: "3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:38:03.137520 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.137482 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7mwj\" (UniqueName: \"kubernetes.io/projected/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-kube-api-access-c7mwj\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:38:03.137520 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.137513 2547 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-config\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:38:03.137520 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.137523 2547 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-oauth-serving-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:38:03.137520 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.137532 2547 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-service-ca\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:38:03.137775 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.137541 2547 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-serving-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:38:03.137775 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.137550 2547 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-console-oauth-config\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:38:03.137775 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.137559 2547 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f-trusted-ca-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:38:03.680007 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.679977 2547 generic.go:356] "Generic (PLEG): container finished" podID="3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" containerID="41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a" exitCode=0 Jul 02 21:38:03.680434 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.680026 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85b755d789-ds4bz" Jul 02 21:38:03.680434 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.680045 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b755d789-ds4bz" event={"ID":"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f","Type":"ContainerDied","Data":"41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a"} Jul 02 21:38:03.680434 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.680085 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85b755d789-ds4bz" event={"ID":"3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f","Type":"ContainerDied","Data":"a13e00b022f230bc1f305f9612662f853d3ecff25c8ed72e137dbc2fc38bf077"} Jul 02 21:38:03.680434 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.680105 2547 scope.go:117] "RemoveContainer" containerID="41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a" Jul 02 21:38:03.688580 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.688554 2547 scope.go:117] "RemoveContainer" containerID="41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a" Jul 02 21:38:03.688836 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:38:03.688819 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a\": container with ID starting with 41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a not found: ID does not exist" containerID="41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a" Jul 02 21:38:03.688894 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.688843 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a"} err="failed to get container status \"41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a\": rpc error: code = NotFound desc = could not find container \"41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a\": container with ID starting with 41ecc14f90224013c333ba9f7745e455a7cca496fe08b25c97a9b67afb563a5a not found: ID does not exist" Jul 02 21:38:03.700651 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.700627 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85b755d789-ds4bz"] Jul 02 21:38:03.703801 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:03.703778 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85b755d789-ds4bz"] Jul 02 21:38:04.007891 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:04.007811 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" path="/var/lib/kubelet/pods/3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f/volumes" Jul 02 21:38:34.414887 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.414856 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fd9cb99db-6blsp"] Jul 02 21:38:34.415421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.415315 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" containerName="console" Jul 02 21:38:34.415421 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.415336 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" containerName="console" Jul 02 21:38:34.415539 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.415451 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f0e50f9-b9ed-490f-96f0-0c934b2bfd4f" containerName="console" Jul 02 21:38:34.418994 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.418974 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.428818 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.428792 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fd9cb99db-6blsp"] Jul 02 21:38:34.497685 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.497651 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-oauth-config\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.497685 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.497682 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-config\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.497966 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.497736 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-oauth-serving-cert\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.497966 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.497780 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-service-ca\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.497966 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.497831 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-trusted-ca-bundle\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.497966 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.497878 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-serving-cert\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.497966 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.497906 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drrqt\" (UniqueName: \"kubernetes.io/projected/30a7fe68-bb12-4769-9fcd-e3c925fd886f-kube-api-access-drrqt\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.598618 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.598568 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-oauth-config\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.598618 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.598617 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-config\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.598870 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.598736 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-oauth-serving-cert\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.598870 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.598770 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-service-ca\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.598870 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.598800 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-trusted-ca-bundle\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.598870 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.598818 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-serving-cert\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.598870 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.598839 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drrqt\" (UniqueName: \"kubernetes.io/projected/30a7fe68-bb12-4769-9fcd-e3c925fd886f-kube-api-access-drrqt\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.599496 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.599470 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-oauth-serving-cert\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.599625 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.599479 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-config\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.599625 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.599533 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-service-ca\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.599800 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.599775 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-trusted-ca-bundle\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.601204 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.601184 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-oauth-config\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.601307 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.601289 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-serving-cert\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.606858 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.606838 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drrqt\" (UniqueName: \"kubernetes.io/projected/30a7fe68-bb12-4769-9fcd-e3c925fd886f-kube-api-access-drrqt\") pod \"console-fd9cb99db-6blsp\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.730428 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.730308 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:34.863515 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:34.863316 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fd9cb99db-6blsp"] Jul 02 21:38:34.866089 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:38:34.866062 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a7fe68_bb12_4769_9fcd_e3c925fd886f.slice/crio-4d006cf60c63df3c0383b2af9589ead3a469e0aaad5ea9437ea968d237647548 WatchSource:0}: Error finding container 4d006cf60c63df3c0383b2af9589ead3a469e0aaad5ea9437ea968d237647548: Status 404 returned error can't find the container with id 4d006cf60c63df3c0383b2af9589ead3a469e0aaad5ea9437ea968d237647548 Jul 02 21:38:35.804573 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:35.804537 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd9cb99db-6blsp" event={"ID":"30a7fe68-bb12-4769-9fcd-e3c925fd886f","Type":"ContainerStarted","Data":"0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847"} Jul 02 21:38:35.804979 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:35.804610 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd9cb99db-6blsp" event={"ID":"30a7fe68-bb12-4769-9fcd-e3c925fd886f","Type":"ContainerStarted","Data":"4d006cf60c63df3c0383b2af9589ead3a469e0aaad5ea9437ea968d237647548"} Jul 02 21:38:35.822679 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:35.822630 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fd9cb99db-6blsp" podStartSLOduration=1.822615238 podStartE2EDuration="1.822615238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:38:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:38:35.821338915 +0000 UTC m=+450.271177542" watchObservedRunningTime="2026-07-02 21:38:35.822615238 +0000 UTC m=+450.272453926" Jul 02 21:38:44.731169 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:44.731132 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:44.731597 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:44.731200 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:44.735753 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:44.735732 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:44.841492 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:44.841466 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:38:44.887216 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:38:44.887186 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85c9d966f6-dlwmw"] Jul 02 21:39:09.911228 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:09.911169 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-85c9d966f6-dlwmw" podUID="e74653a9-f041-47a7-98f9-1f289b90b809" containerName="console" containerID="cri-o://68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a" gracePeriod=15 Jul 02 21:39:10.164553 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.164486 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:39:10.196766 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.196737 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8gc6\" (UniqueName: \"kubernetes.io/projected/e74653a9-f041-47a7-98f9-1f289b90b809-kube-api-access-z8gc6\") pod \"e74653a9-f041-47a7-98f9-1f289b90b809\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " Jul 02 21:39:10.196945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.196797 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-serving-cert\") pod \"e74653a9-f041-47a7-98f9-1f289b90b809\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " Jul 02 21:39:10.196945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.196830 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-console-config\") pod \"e74653a9-f041-47a7-98f9-1f289b90b809\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " Jul 02 21:39:10.196945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.196876 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-trusted-ca-bundle\") pod \"e74653a9-f041-47a7-98f9-1f289b90b809\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " Jul 02 21:39:10.196945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.196908 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-service-ca\") pod \"e74653a9-f041-47a7-98f9-1f289b90b809\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " Jul 02 21:39:10.197152 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.197015 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-oauth-config\") pod \"e74653a9-f041-47a7-98f9-1f289b90b809\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " Jul 02 21:39:10.197152 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.197065 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-oauth-serving-cert\") pod \"e74653a9-f041-47a7-98f9-1f289b90b809\" (UID: \"e74653a9-f041-47a7-98f9-1f289b90b809\") " Jul 02 21:39:10.197340 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.197292 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-console-config" (OuterVolumeSpecName: "console-config") pod "e74653a9-f041-47a7-98f9-1f289b90b809" (UID: "e74653a9-f041-47a7-98f9-1f289b90b809"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:39:10.197340 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.197295 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-service-ca" (OuterVolumeSpecName: "service-ca") pod "e74653a9-f041-47a7-98f9-1f289b90b809" (UID: "e74653a9-f041-47a7-98f9-1f289b90b809"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:39:10.197664 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.197436 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e74653a9-f041-47a7-98f9-1f289b90b809" (UID: "e74653a9-f041-47a7-98f9-1f289b90b809"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:39:10.197664 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.197595 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e74653a9-f041-47a7-98f9-1f289b90b809" (UID: "e74653a9-f041-47a7-98f9-1f289b90b809"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:39:10.199203 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.199180 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e74653a9-f041-47a7-98f9-1f289b90b809-kube-api-access-z8gc6" (OuterVolumeSpecName: "kube-api-access-z8gc6") pod "e74653a9-f041-47a7-98f9-1f289b90b809" (UID: "e74653a9-f041-47a7-98f9-1f289b90b809"). InnerVolumeSpecName "kube-api-access-z8gc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:39:10.199315 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.199211 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e74653a9-f041-47a7-98f9-1f289b90b809" (UID: "e74653a9-f041-47a7-98f9-1f289b90b809"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:39:10.199529 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.199505 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e74653a9-f041-47a7-98f9-1f289b90b809" (UID: "e74653a9-f041-47a7-98f9-1f289b90b809"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:39:10.298570 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.298533 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8gc6\" (UniqueName: \"kubernetes.io/projected/e74653a9-f041-47a7-98f9-1f289b90b809-kube-api-access-z8gc6\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:39:10.298570 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.298561 2547 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-serving-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:39:10.298570 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.298572 2547 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-console-config\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:39:10.298570 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.298581 2547 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-trusted-ca-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:39:10.298837 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.298589 2547 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-service-ca\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:39:10.298837 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.298598 2547 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e74653a9-f041-47a7-98f9-1f289b90b809-console-oauth-config\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:39:10.298837 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.298606 2547 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e74653a9-f041-47a7-98f9-1f289b90b809-oauth-serving-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:39:10.942190 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.942153 2547 generic.go:356] "Generic (PLEG): container finished" podID="e74653a9-f041-47a7-98f9-1f289b90b809" containerID="68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a" exitCode=0 Jul 02 21:39:10.942681 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.942199 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85c9d966f6-dlwmw" Jul 02 21:39:10.942681 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.942213 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c9d966f6-dlwmw" event={"ID":"e74653a9-f041-47a7-98f9-1f289b90b809","Type":"ContainerDied","Data":"68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a"} Jul 02 21:39:10.942681 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.942263 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85c9d966f6-dlwmw" event={"ID":"e74653a9-f041-47a7-98f9-1f289b90b809","Type":"ContainerDied","Data":"6cafd0a9832bda81278ef539c041f407b43b3dbdda65bc21ef1729e400855156"} Jul 02 21:39:10.942681 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.942294 2547 scope.go:117] "RemoveContainer" containerID="68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a" Jul 02 21:39:10.954496 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.954477 2547 scope.go:117] "RemoveContainer" containerID="68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a" Jul 02 21:39:10.954778 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:39:10.954759 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a\": container with ID starting with 68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a not found: ID does not exist" containerID="68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a" Jul 02 21:39:10.954829 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.954787 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a"} err="failed to get container status \"68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a\": rpc error: code = NotFound desc = could not find container \"68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a\": container with ID starting with 68f79173afbee451a0f16b5c7a2e39b60bb44ac2943a6af553e3c8b56fa6834a not found: ID does not exist" Jul 02 21:39:10.966091 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.966066 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85c9d966f6-dlwmw"] Jul 02 21:39:10.970088 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:10.970062 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85c9d966f6-dlwmw"] Jul 02 21:39:12.007931 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:39:12.007900 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e74653a9-f041-47a7-98f9-1f289b90b809" path="/var/lib/kubelet/pods/e74653a9-f041-47a7-98f9-1f289b90b809/volumes" Jul 02 21:41:05.895788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:41:05.895759 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:41:05.895788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:41:05.895784 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:41:05.903394 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:41:05.903358 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:41:05.903628 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:41:05.903606 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:42:53.995829 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:53.995797 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d6875dbb7-xpjxk"] Jul 02 21:42:53.996403 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:53.996119 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e74653a9-f041-47a7-98f9-1f289b90b809" containerName="console" Jul 02 21:42:53.996403 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:53.996134 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e74653a9-f041-47a7-98f9-1f289b90b809" containerName="console" Jul 02 21:42:53.996403 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:53.996218 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e74653a9-f041-47a7-98f9-1f289b90b809" containerName="console" Jul 02 21:42:53.999540 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:53.999519 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.018602 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.018574 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d6875dbb7-xpjxk"] Jul 02 21:42:54.036245 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.036203 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-oauth-serving-cert\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.036245 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.036244 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-console-serving-cert\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.036503 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.036289 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-trusted-ca-bundle\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.036503 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.036304 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd5tq\" (UniqueName: \"kubernetes.io/projected/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-kube-api-access-wd5tq\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.036503 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.036412 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-console-oauth-config\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.036503 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.036492 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-console-config\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.036681 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.036533 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-service-ca\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.137597 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.137560 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-service-ca\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.137806 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.137614 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-oauth-serving-cert\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.137806 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.137738 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-console-serving-cert\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.137806 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.137802 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-trusted-ca-bundle\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.137965 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.137826 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd5tq\" (UniqueName: \"kubernetes.io/projected/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-kube-api-access-wd5tq\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.137965 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.137884 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-console-oauth-config\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.137965 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.137954 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-console-config\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.138418 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.138370 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-oauth-serving-cert\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.138418 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.138369 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-service-ca\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.138654 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.138634 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-console-config\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.138965 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.138945 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-trusted-ca-bundle\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.140304 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.140285 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-console-serving-cert\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.140464 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.140443 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-console-oauth-config\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.145622 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.145598 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd5tq\" (UniqueName: \"kubernetes.io/projected/6f0babc3-5f42-4ebc-bd16-542d3e5c0429-kube-api-access-wd5tq\") pod \"console-d6875dbb7-xpjxk\" (UID: \"6f0babc3-5f42-4ebc-bd16-542d3e5c0429\") " pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.311052 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.311027 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:42:54.445777 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.445749 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d6875dbb7-xpjxk"] Jul 02 21:42:54.450460 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:42:54.450431 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f0babc3_5f42_4ebc_bd16_542d3e5c0429.slice/crio-c4a80bf49d4aa2c36c907ae34533a9dfc854ee5df87fbe704504cbbdc14a9b9a WatchSource:0}: Error finding container c4a80bf49d4aa2c36c907ae34533a9dfc854ee5df87fbe704504cbbdc14a9b9a: Status 404 returned error can't find the container with id c4a80bf49d4aa2c36c907ae34533a9dfc854ee5df87fbe704504cbbdc14a9b9a Jul 02 21:42:54.815289 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.815258 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d6875dbb7-xpjxk" event={"ID":"6f0babc3-5f42-4ebc-bd16-542d3e5c0429","Type":"ContainerStarted","Data":"be3167fb997f96eec7004f11c924e133a0a3c545b2c9b74b637372e6e592caf4"} Jul 02 21:42:54.815500 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.815335 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d6875dbb7-xpjxk" event={"ID":"6f0babc3-5f42-4ebc-bd16-542d3e5c0429","Type":"ContainerStarted","Data":"c4a80bf49d4aa2c36c907ae34533a9dfc854ee5df87fbe704504cbbdc14a9b9a"} Jul 02 21:42:54.833822 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:42:54.833763 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d6875dbb7-xpjxk" podStartSLOduration=1.833741708 podStartE2EDuration="1.833741708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:42:54.832045065 +0000 UTC m=+709.281883693" watchObservedRunningTime="2026-07-02 21:42:54.833741708 +0000 UTC m=+709.283580335" Jul 02 21:43:04.311260 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:04.311220 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:43:04.311774 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:04.311311 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:43:04.315889 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:04.315862 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:43:04.857793 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:04.857765 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d6875dbb7-xpjxk" Jul 02 21:43:04.907411 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:04.907361 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fd9cb99db-6blsp"] Jul 02 21:43:09.333780 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.333750 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll"] Jul 02 21:43:09.337191 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.337175 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.339736 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.339714 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nks98\"" Jul 02 21:43:09.339827 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.339714 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 21:43:09.340814 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.340797 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 21:43:09.346276 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.346256 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll"] Jul 02 21:43:09.475052 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.474989 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.475241 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.475090 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.475241 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.475152 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbd99\" (UniqueName: \"kubernetes.io/projected/3c5eb175-2872-46e5-8536-59b631ac34c3-kube-api-access-bbd99\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.576275 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.576237 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.576454 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.576291 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.576454 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.576333 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbd99\" (UniqueName: \"kubernetes.io/projected/3c5eb175-2872-46e5-8536-59b631ac34c3-kube-api-access-bbd99\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.576646 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.576624 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.576711 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.576691 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.585401 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.585321 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbd99\" (UniqueName: \"kubernetes.io/projected/3c5eb175-2872-46e5-8536-59b631ac34c3-kube-api-access-bbd99\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.646493 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.646466 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:09.775556 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.775533 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll"] Jul 02 21:43:09.777741 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:43:09.777709 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5eb175_2872_46e5_8536_59b631ac34c3.slice/crio-c4f625d3ec875ea96868711a3f5fccd20a107a676033ba4ccbf1459002c5397d WatchSource:0}: Error finding container c4f625d3ec875ea96868711a3f5fccd20a107a676033ba4ccbf1459002c5397d: Status 404 returned error can't find the container with id c4f625d3ec875ea96868711a3f5fccd20a107a676033ba4ccbf1459002c5397d Jul 02 21:43:09.779699 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.779680 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:43:09.875182 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:09.875101 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" event={"ID":"3c5eb175-2872-46e5-8536-59b631ac34c3","Type":"ContainerStarted","Data":"c4f625d3ec875ea96868711a3f5fccd20a107a676033ba4ccbf1459002c5397d"} Jul 02 21:43:16.907207 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:16.907115 2547 generic.go:356] "Generic (PLEG): container finished" podID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerID="bab422c6e85c321abae6cc74e78554fb868954fc24f3fc2e24c0e5dda38616dc" exitCode=0 Jul 02 21:43:16.907657 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:16.907197 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" event={"ID":"3c5eb175-2872-46e5-8536-59b631ac34c3","Type":"ContainerDied","Data":"bab422c6e85c321abae6cc74e78554fb868954fc24f3fc2e24c0e5dda38616dc"} Jul 02 21:43:26.950162 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:26.950124 2547 generic.go:356] "Generic (PLEG): container finished" podID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerID="1cd25c36a3bfda776b2555a98c69a7989dbe86ae9f1570ee5a9fc1825af2b9a1" exitCode=0 Jul 02 21:43:26.950615 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:26.950211 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" event={"ID":"3c5eb175-2872-46e5-8536-59b631ac34c3","Type":"ContainerDied","Data":"1cd25c36a3bfda776b2555a98c69a7989dbe86ae9f1570ee5a9fc1825af2b9a1"} Jul 02 21:43:29.931278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:29.931231 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-fd9cb99db-6blsp" podUID="30a7fe68-bb12-4769-9fcd-e3c925fd886f" containerName="console" containerID="cri-o://0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847" gracePeriod=15 Jul 02 21:43:30.194091 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.194033 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:43:30.367449 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.367421 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-trusted-ca-bundle\") pod \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " Jul 02 21:43:30.367623 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.367468 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-oauth-config\") pod \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " Jul 02 21:43:30.367623 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.367508 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-service-ca\") pod \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " Jul 02 21:43:30.367623 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.367555 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-serving-cert\") pod \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " Jul 02 21:43:30.367623 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.367591 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-config\") pod \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " Jul 02 21:43:30.367623 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.367622 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drrqt\" (UniqueName: \"kubernetes.io/projected/30a7fe68-bb12-4769-9fcd-e3c925fd886f-kube-api-access-drrqt\") pod \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " Jul 02 21:43:30.367856 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.367659 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-oauth-serving-cert\") pod \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\" (UID: \"30a7fe68-bb12-4769-9fcd-e3c925fd886f\") " Jul 02 21:43:30.367916 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.367882 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "30a7fe68-bb12-4769-9fcd-e3c925fd886f" (UID: "30a7fe68-bb12-4769-9fcd-e3c925fd886f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:43:30.368065 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.368010 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-service-ca" (OuterVolumeSpecName: "service-ca") pod "30a7fe68-bb12-4769-9fcd-e3c925fd886f" (UID: "30a7fe68-bb12-4769-9fcd-e3c925fd886f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:43:30.368225 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.368194 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-config" (OuterVolumeSpecName: "console-config") pod "30a7fe68-bb12-4769-9fcd-e3c925fd886f" (UID: "30a7fe68-bb12-4769-9fcd-e3c925fd886f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:43:30.368500 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.368457 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "30a7fe68-bb12-4769-9fcd-e3c925fd886f" (UID: "30a7fe68-bb12-4769-9fcd-e3c925fd886f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:43:30.369908 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.369874 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "30a7fe68-bb12-4769-9fcd-e3c925fd886f" (UID: "30a7fe68-bb12-4769-9fcd-e3c925fd886f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:43:30.370019 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.369951 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a7fe68-bb12-4769-9fcd-e3c925fd886f-kube-api-access-drrqt" (OuterVolumeSpecName: "kube-api-access-drrqt") pod "30a7fe68-bb12-4769-9fcd-e3c925fd886f" (UID: "30a7fe68-bb12-4769-9fcd-e3c925fd886f"). InnerVolumeSpecName "kube-api-access-drrqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:43:30.370322 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.370296 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "30a7fe68-bb12-4769-9fcd-e3c925fd886f" (UID: "30a7fe68-bb12-4769-9fcd-e3c925fd886f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:43:30.469167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.469087 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drrqt\" (UniqueName: \"kubernetes.io/projected/30a7fe68-bb12-4769-9fcd-e3c925fd886f-kube-api-access-drrqt\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:30.469167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.469118 2547 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-oauth-serving-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:30.469167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.469131 2547 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-trusted-ca-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:30.469167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.469143 2547 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-oauth-config\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:30.469167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.469156 2547 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-service-ca\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:30.469167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.469168 2547 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-serving-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:30.469543 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.469180 2547 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30a7fe68-bb12-4769-9fcd-e3c925fd886f-console-config\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:30.969409 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.969361 2547 generic.go:356] "Generic (PLEG): container finished" podID="30a7fe68-bb12-4769-9fcd-e3c925fd886f" containerID="0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847" exitCode=0 Jul 02 21:43:30.969409 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.969418 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fd9cb99db-6blsp" Jul 02 21:43:30.969856 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.969453 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd9cb99db-6blsp" event={"ID":"30a7fe68-bb12-4769-9fcd-e3c925fd886f","Type":"ContainerDied","Data":"0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847"} Jul 02 21:43:30.969856 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.969506 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fd9cb99db-6blsp" event={"ID":"30a7fe68-bb12-4769-9fcd-e3c925fd886f","Type":"ContainerDied","Data":"4d006cf60c63df3c0383b2af9589ead3a469e0aaad5ea9437ea968d237647548"} Jul 02 21:43:30.969856 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.969528 2547 scope.go:117] "RemoveContainer" containerID="0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847" Jul 02 21:43:30.983422 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.983402 2547 scope.go:117] "RemoveContainer" containerID="0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847" Jul 02 21:43:30.983694 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:43:30.983673 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847\": container with ID starting with 0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847 not found: ID does not exist" containerID="0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847" Jul 02 21:43:30.983767 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.983703 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847"} err="failed to get container status \"0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847\": rpc error: code = NotFound desc = could not find container \"0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847\": container with ID starting with 0160ab2f62e5913664fd453a6cf6813bdf97d3689c819a05a11fa672a81fe847 not found: ID does not exist" Jul 02 21:43:30.992953 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.992930 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fd9cb99db-6blsp"] Jul 02 21:43:30.996611 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:30.996592 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fd9cb99db-6blsp"] Jul 02 21:43:32.008558 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:32.008519 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30a7fe68-bb12-4769-9fcd-e3c925fd886f" path="/var/lib/kubelet/pods/30a7fe68-bb12-4769-9fcd-e3c925fd886f/volumes" Jul 02 21:43:34.992337 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:34.992249 2547 generic.go:356] "Generic (PLEG): container finished" podID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerID="47ce253d86ea71f136c17b1db2c196d9a7d7154f69e4c76d40144ada2664ceb1" exitCode=0 Jul 02 21:43:34.992337 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:34.992292 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" event={"ID":"3c5eb175-2872-46e5-8536-59b631ac34c3","Type":"ContainerDied","Data":"47ce253d86ea71f136c17b1db2c196d9a7d7154f69e4c76d40144ada2664ceb1"} Jul 02 21:43:36.123777 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.123755 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:36.210587 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.210558 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-bundle\") pod \"3c5eb175-2872-46e5-8536-59b631ac34c3\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " Jul 02 21:43:36.211183 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.211162 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-bundle" (OuterVolumeSpecName: "bundle") pod "3c5eb175-2872-46e5-8536-59b631ac34c3" (UID: "3c5eb175-2872-46e5-8536-59b631ac34c3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:43:36.310970 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.310945 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-util\") pod \"3c5eb175-2872-46e5-8536-59b631ac34c3\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " Jul 02 21:43:36.311118 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.310981 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbd99\" (UniqueName: \"kubernetes.io/projected/3c5eb175-2872-46e5-8536-59b631ac34c3-kube-api-access-bbd99\") pod \"3c5eb175-2872-46e5-8536-59b631ac34c3\" (UID: \"3c5eb175-2872-46e5-8536-59b631ac34c3\") " Jul 02 21:43:36.311188 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.311120 2547 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:36.313132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.313106 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c5eb175-2872-46e5-8536-59b631ac34c3-kube-api-access-bbd99" (OuterVolumeSpecName: "kube-api-access-bbd99") pod "3c5eb175-2872-46e5-8536-59b631ac34c3" (UID: "3c5eb175-2872-46e5-8536-59b631ac34c3"). InnerVolumeSpecName "kube-api-access-bbd99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:43:36.315745 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.315724 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-util" (OuterVolumeSpecName: "util") pod "3c5eb175-2872-46e5-8536-59b631ac34c3" (UID: "3c5eb175-2872-46e5-8536-59b631ac34c3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:43:36.411803 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.411776 2547 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c5eb175-2872-46e5-8536-59b631ac34c3-util\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:36.411803 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:36.411800 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbd99\" (UniqueName: \"kubernetes.io/projected/3c5eb175-2872-46e5-8536-59b631ac34c3-kube-api-access-bbd99\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:37.002835 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:37.002809 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" Jul 02 21:43:37.003010 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:37.002807 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ll9ll" event={"ID":"3c5eb175-2872-46e5-8536-59b631ac34c3","Type":"ContainerDied","Data":"c4f625d3ec875ea96868711a3f5fccd20a107a676033ba4ccbf1459002c5397d"} Jul 02 21:43:37.003010 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:37.002925 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4f625d3ec875ea96868711a3f5fccd20a107a676033ba4ccbf1459002c5397d" Jul 02 21:43:46.334501 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334465 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm"] Jul 02 21:43:46.334878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334775 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerName="extract" Jul 02 21:43:46.334878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334787 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerName="extract" Jul 02 21:43:46.334878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334800 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerName="util" Jul 02 21:43:46.334878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334806 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerName="util" Jul 02 21:43:46.334878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334822 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerName="pull" Jul 02 21:43:46.334878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334829 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerName="pull" Jul 02 21:43:46.334878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334839 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a7fe68-bb12-4769-9fcd-e3c925fd886f" containerName="console" Jul 02 21:43:46.334878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334844 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a7fe68-bb12-4769-9fcd-e3c925fd886f" containerName="console" Jul 02 21:43:46.335109 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334893 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a7fe68-bb12-4769-9fcd-e3c925fd886f" containerName="console" Jul 02 21:43:46.335109 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.334900 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c5eb175-2872-46e5-8536-59b631ac34c3" containerName="extract" Jul 02 21:43:46.338095 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.338079 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.340763 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.340734 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nks98\"" Jul 02 21:43:46.340879 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.340764 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 21:43:46.341612 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.341592 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 21:43:46.347668 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.347648 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm"] Jul 02 21:43:46.390586 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.390555 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5278\" (UniqueName: \"kubernetes.io/projected/f1547ce7-08a3-48e2-b597-d4190d1257f0-kube-api-access-w5278\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.390758 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.390592 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.390758 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.390667 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.491457 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.491410 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5278\" (UniqueName: \"kubernetes.io/projected/f1547ce7-08a3-48e2-b597-d4190d1257f0-kube-api-access-w5278\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.491457 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.491464 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.491711 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.491512 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.491949 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.491928 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.491949 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.491943 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.500564 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.500534 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5278\" (UniqueName: \"kubernetes.io/projected/f1547ce7-08a3-48e2-b597-d4190d1257f0-kube-api-access-w5278\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.648116 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.648032 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:46.787672 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:46.787648 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm"] Jul 02 21:43:46.789938 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:43:46.789914 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1547ce7_08a3_48e2_b597_d4190d1257f0.slice/crio-4162e4608ab167028674fe36ed6d1beaef6028439c7537c72210302b0cb2c961 WatchSource:0}: Error finding container 4162e4608ab167028674fe36ed6d1beaef6028439c7537c72210302b0cb2c961: Status 404 returned error can't find the container with id 4162e4608ab167028674fe36ed6d1beaef6028439c7537c72210302b0cb2c961 Jul 02 21:43:47.044008 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:47.043926 2547 generic.go:356] "Generic (PLEG): container finished" podID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerID="6eddea4c516ddae8305d351a2211ad6ab27a5fbd4a0a220ad5374fe4838207a6" exitCode=0 Jul 02 21:43:47.044008 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:47.043970 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" event={"ID":"f1547ce7-08a3-48e2-b597-d4190d1257f0","Type":"ContainerDied","Data":"6eddea4c516ddae8305d351a2211ad6ab27a5fbd4a0a220ad5374fe4838207a6"} Jul 02 21:43:47.044192 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:47.044021 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" event={"ID":"f1547ce7-08a3-48e2-b597-d4190d1257f0","Type":"ContainerStarted","Data":"4162e4608ab167028674fe36ed6d1beaef6028439c7537c72210302b0cb2c961"} Jul 02 21:43:49.055109 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.055077 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" event={"ID":"f1547ce7-08a3-48e2-b597-d4190d1257f0","Type":"ContainerStarted","Data":"b4b8afc3dff82ec9807e8e90e383c3e6ea6fd67f336565298c030242c86cda16"} Jul 02 21:43:49.180559 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.180524 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw"] Jul 02 21:43:49.183928 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.183910 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" Jul 02 21:43:49.186604 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.186583 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 21:43:49.186702 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.186595 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-5lqkq\"" Jul 02 21:43:49.186702 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.186678 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 21:43:49.192683 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.192659 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw"] Jul 02 21:43:49.216606 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.216573 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssdr5\" (UniqueName: \"kubernetes.io/projected/f183501e-2fdc-4655-af44-6a90f3d54c15-kube-api-access-ssdr5\") pod \"cert-manager-cainjector-559cdd8c87-gpgrw\" (UID: \"f183501e-2fdc-4655-af44-6a90f3d54c15\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" Jul 02 21:43:49.216758 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.216652 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f183501e-2fdc-4655-af44-6a90f3d54c15-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gpgrw\" (UID: \"f183501e-2fdc-4655-af44-6a90f3d54c15\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" Jul 02 21:43:49.317090 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.317053 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssdr5\" (UniqueName: \"kubernetes.io/projected/f183501e-2fdc-4655-af44-6a90f3d54c15-kube-api-access-ssdr5\") pod \"cert-manager-cainjector-559cdd8c87-gpgrw\" (UID: \"f183501e-2fdc-4655-af44-6a90f3d54c15\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" Jul 02 21:43:49.317286 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.317139 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f183501e-2fdc-4655-af44-6a90f3d54c15-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gpgrw\" (UID: \"f183501e-2fdc-4655-af44-6a90f3d54c15\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" Jul 02 21:43:49.325305 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.325282 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssdr5\" (UniqueName: \"kubernetes.io/projected/f183501e-2fdc-4655-af44-6a90f3d54c15-kube-api-access-ssdr5\") pod \"cert-manager-cainjector-559cdd8c87-gpgrw\" (UID: \"f183501e-2fdc-4655-af44-6a90f3d54c15\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" Jul 02 21:43:49.325442 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.325328 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f183501e-2fdc-4655-af44-6a90f3d54c15-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-gpgrw\" (UID: \"f183501e-2fdc-4655-af44-6a90f3d54c15\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" Jul 02 21:43:49.504274 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.504227 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" Jul 02 21:43:49.629171 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:49.629144 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw"] Jul 02 21:43:49.632250 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:43:49.632219 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf183501e_2fdc_4655_af44_6a90f3d54c15.slice/crio-fcd71aab9d86b6ec8d3b100bb1f317f40d0a235e57b6347433dd08d903ec6dc3 WatchSource:0}: Error finding container fcd71aab9d86b6ec8d3b100bb1f317f40d0a235e57b6347433dd08d903ec6dc3: Status 404 returned error can't find the container with id fcd71aab9d86b6ec8d3b100bb1f317f40d0a235e57b6347433dd08d903ec6dc3 Jul 02 21:43:50.060669 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:50.060630 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" event={"ID":"f183501e-2fdc-4655-af44-6a90f3d54c15","Type":"ContainerStarted","Data":"fcd71aab9d86b6ec8d3b100bb1f317f40d0a235e57b6347433dd08d903ec6dc3"} Jul 02 21:43:50.062476 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:50.062452 2547 generic.go:356] "Generic (PLEG): container finished" podID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerID="b4b8afc3dff82ec9807e8e90e383c3e6ea6fd67f336565298c030242c86cda16" exitCode=0 Jul 02 21:43:50.062593 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:50.062535 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" event={"ID":"f1547ce7-08a3-48e2-b597-d4190d1257f0","Type":"ContainerDied","Data":"b4b8afc3dff82ec9807e8e90e383c3e6ea6fd67f336565298c030242c86cda16"} Jul 02 21:43:51.070661 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:51.070629 2547 generic.go:356] "Generic (PLEG): container finished" podID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerID="8dde4cbc0141e246c2030f90bb9113a9638144d635e8a3f5e4d0a8bc17ecc6f0" exitCode=0 Jul 02 21:43:51.071082 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:51.070754 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" event={"ID":"f1547ce7-08a3-48e2-b597-d4190d1257f0","Type":"ContainerDied","Data":"8dde4cbc0141e246c2030f90bb9113a9638144d635e8a3f5e4d0a8bc17ecc6f0"} Jul 02 21:43:52.220775 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.220751 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:52.346280 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.346190 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-bundle\") pod \"f1547ce7-08a3-48e2-b597-d4190d1257f0\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " Jul 02 21:43:52.346280 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.346258 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5278\" (UniqueName: \"kubernetes.io/projected/f1547ce7-08a3-48e2-b597-d4190d1257f0-kube-api-access-w5278\") pod \"f1547ce7-08a3-48e2-b597-d4190d1257f0\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " Jul 02 21:43:52.346480 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.346289 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-util\") pod \"f1547ce7-08a3-48e2-b597-d4190d1257f0\" (UID: \"f1547ce7-08a3-48e2-b597-d4190d1257f0\") " Jul 02 21:43:52.346717 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.346682 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-bundle" (OuterVolumeSpecName: "bundle") pod "f1547ce7-08a3-48e2-b597-d4190d1257f0" (UID: "f1547ce7-08a3-48e2-b597-d4190d1257f0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:43:52.348720 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.348694 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1547ce7-08a3-48e2-b597-d4190d1257f0-kube-api-access-w5278" (OuterVolumeSpecName: "kube-api-access-w5278") pod "f1547ce7-08a3-48e2-b597-d4190d1257f0" (UID: "f1547ce7-08a3-48e2-b597-d4190d1257f0"). InnerVolumeSpecName "kube-api-access-w5278". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:43:52.352869 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.352834 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-util" (OuterVolumeSpecName: "util") pod "f1547ce7-08a3-48e2-b597-d4190d1257f0" (UID: "f1547ce7-08a3-48e2-b597-d4190d1257f0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:43:52.447744 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.447706 2547 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:52.447744 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.447741 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5278\" (UniqueName: \"kubernetes.io/projected/f1547ce7-08a3-48e2-b597-d4190d1257f0-kube-api-access-w5278\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:52.447948 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:52.447756 2547 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f1547ce7-08a3-48e2-b597-d4190d1257f0-util\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:43:53.081665 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:53.081627 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" event={"ID":"f183501e-2fdc-4655-af44-6a90f3d54c15","Type":"ContainerStarted","Data":"5c91aca46f5c5a204fc86f51e33eeae41f88b0f54ac9b5f35fe286e63ff065a5"} Jul 02 21:43:53.083510 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:53.083490 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" Jul 02 21:43:53.083620 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:53.083519 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fcbnfm" event={"ID":"f1547ce7-08a3-48e2-b597-d4190d1257f0","Type":"ContainerDied","Data":"4162e4608ab167028674fe36ed6d1beaef6028439c7537c72210302b0cb2c961"} Jul 02 21:43:53.083620 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:53.083553 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4162e4608ab167028674fe36ed6d1beaef6028439c7537c72210302b0cb2c961" Jul 02 21:43:53.101329 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:53.101285 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-gpgrw" podStartSLOduration=1.586047913 podStartE2EDuration="4.101270419s" totalImagesPullingTime="2.515222506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:43:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:43:53.099327352 +0000 UTC m=+767.549165976" watchObservedRunningTime="2026-07-02 21:43:53.101270419 +0000 UTC m=+767.551109042" Jul 02 21:43:56.641601 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.641569 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl"] Jul 02 21:43:56.641977 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.641863 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerName="util" Jul 02 21:43:56.641977 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.641874 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerName="util" Jul 02 21:43:56.641977 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.641888 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerName="pull" Jul 02 21:43:56.641977 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.641893 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerName="pull" Jul 02 21:43:56.641977 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.641907 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerName="extract" Jul 02 21:43:56.641977 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.641913 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerName="extract" Jul 02 21:43:56.641977 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.641959 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1547ce7-08a3-48e2-b597-d4190d1257f0" containerName="extract" Jul 02 21:43:56.648091 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.648074 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" Jul 02 21:43:56.651310 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.651265 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-d7gn2\"" Jul 02 21:43:56.651824 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.651339 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 21:43:56.651824 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.651402 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 21:43:56.652295 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.652272 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl"] Jul 02 21:43:56.783734 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.783699 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h42hv\" (UniqueName: \"kubernetes.io/projected/688f550f-2a9e-41b0-a72d-770a2fc92847-kube-api-access-h42hv\") pod \"openshift-lws-operator-fd8ccff4c-gvldl\" (UID: \"688f550f-2a9e-41b0-a72d-770a2fc92847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" Jul 02 21:43:56.784022 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.783999 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/688f550f-2a9e-41b0-a72d-770a2fc92847-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gvldl\" (UID: \"688f550f-2a9e-41b0-a72d-770a2fc92847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" Jul 02 21:43:56.885482 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.885432 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h42hv\" (UniqueName: \"kubernetes.io/projected/688f550f-2a9e-41b0-a72d-770a2fc92847-kube-api-access-h42hv\") pod \"openshift-lws-operator-fd8ccff4c-gvldl\" (UID: \"688f550f-2a9e-41b0-a72d-770a2fc92847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" Jul 02 21:43:56.885668 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.885585 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/688f550f-2a9e-41b0-a72d-770a2fc92847-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gvldl\" (UID: \"688f550f-2a9e-41b0-a72d-770a2fc92847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" Jul 02 21:43:56.885910 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.885895 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/688f550f-2a9e-41b0-a72d-770a2fc92847-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gvldl\" (UID: \"688f550f-2a9e-41b0-a72d-770a2fc92847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" Jul 02 21:43:56.894014 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.893952 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h42hv\" (UniqueName: \"kubernetes.io/projected/688f550f-2a9e-41b0-a72d-770a2fc92847-kube-api-access-h42hv\") pod \"openshift-lws-operator-fd8ccff4c-gvldl\" (UID: \"688f550f-2a9e-41b0-a72d-770a2fc92847\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" Jul 02 21:43:56.958029 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:56.958000 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" Jul 02 21:43:57.086155 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:57.085829 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl"] Jul 02 21:43:57.088288 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:43:57.088255 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod688f550f_2a9e_41b0_a72d_770a2fc92847.slice/crio-d7ecbd8116b76e761791382f97301577c1022d1b6718419b84f24a36c0996e26 WatchSource:0}: Error finding container d7ecbd8116b76e761791382f97301577c1022d1b6718419b84f24a36c0996e26: Status 404 returned error can't find the container with id d7ecbd8116b76e761791382f97301577c1022d1b6718419b84f24a36c0996e26 Jul 02 21:43:57.101664 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:57.101639 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" event={"ID":"688f550f-2a9e-41b0-a72d-770a2fc92847","Type":"ContainerStarted","Data":"d7ecbd8116b76e761791382f97301577c1022d1b6718419b84f24a36c0996e26"} Jul 02 21:43:59.112580 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:59.112544 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" event={"ID":"688f550f-2a9e-41b0-a72d-770a2fc92847","Type":"ContainerStarted","Data":"45ed3dd5f0175d4e88a7e811aa1736673892273250c216b7c9d06f6860558278"} Jul 02 21:43:59.128988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:43:59.128931 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gvldl" podStartSLOduration=1.325968098 podStartE2EDuration="3.12891546s" totalImagesPullingTime="1.802947362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:43:59.127414847 +0000 UTC m=+773.577253465" watchObservedRunningTime="2026-07-02 21:43:59.12891546 +0000 UTC m=+773.578754084" Jul 02 21:44:02.088824 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.088790 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn"] Jul 02 21:44:02.113889 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.113857 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn"] Jul 02 21:44:02.114033 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.113949 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.116659 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.116638 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 02 21:44:02.117663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.117649 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 02 21:44:02.117831 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.117815 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nks98\"" Jul 02 21:44:02.235744 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.235709 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.235744 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.235748 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.235960 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.235778 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p66z9\" (UniqueName: \"kubernetes.io/projected/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-kube-api-access-p66z9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.336333 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.336298 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p66z9\" (UniqueName: \"kubernetes.io/projected/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-kube-api-access-p66z9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.336540 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.336359 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.336540 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.336417 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.336851 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.336831 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.336889 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.336844 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.349649 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.349596 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p66z9\" (UniqueName: \"kubernetes.io/projected/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-kube-api-access-p66z9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.423177 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.423146 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:02.562610 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:02.562569 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn"] Jul 02 21:44:02.564896 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:44:02.564869 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0515d7_8994_40c1_81b1_e9e15bbb1d5e.slice/crio-d5e3d469b04d96f9b7f951d1d65ee044996f8a9e3afa9f3b9214261e68b510b6 WatchSource:0}: Error finding container d5e3d469b04d96f9b7f951d1d65ee044996f8a9e3afa9f3b9214261e68b510b6: Status 404 returned error can't find the container with id d5e3d469b04d96f9b7f951d1d65ee044996f8a9e3afa9f3b9214261e68b510b6 Jul 02 21:44:03.132210 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:03.131100 2547 generic.go:356] "Generic (PLEG): container finished" podID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerID="be737c13233b8f1ca6a63b911d666df7013ba7ab906f0a4eb8fce58f42b3a83d" exitCode=0 Jul 02 21:44:03.132210 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:03.131181 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" event={"ID":"da0515d7-8994-40c1-81b1-e9e15bbb1d5e","Type":"ContainerDied","Data":"be737c13233b8f1ca6a63b911d666df7013ba7ab906f0a4eb8fce58f42b3a83d"} Jul 02 21:44:03.132210 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:03.131275 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" event={"ID":"da0515d7-8994-40c1-81b1-e9e15bbb1d5e","Type":"ContainerStarted","Data":"d5e3d469b04d96f9b7f951d1d65ee044996f8a9e3afa9f3b9214261e68b510b6"} Jul 02 21:44:05.144583 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:05.144547 2547 generic.go:356] "Generic (PLEG): container finished" podID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerID="34ab4d63f08f180b3b02d0badfe112dd0c521e6004df033994b6067d8a428948" exitCode=0 Jul 02 21:44:05.144965 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:05.144605 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" event={"ID":"da0515d7-8994-40c1-81b1-e9e15bbb1d5e","Type":"ContainerDied","Data":"34ab4d63f08f180b3b02d0badfe112dd0c521e6004df033994b6067d8a428948"} Jul 02 21:44:06.151360 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:06.151319 2547 generic.go:356] "Generic (PLEG): container finished" podID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerID="2b7b7920ab89185ca063e12f253f0177efe8c41bfa6648959b01b0e816318ea0" exitCode=0 Jul 02 21:44:06.151769 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:06.151407 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" event={"ID":"da0515d7-8994-40c1-81b1-e9e15bbb1d5e","Type":"ContainerDied","Data":"2b7b7920ab89185ca063e12f253f0177efe8c41bfa6648959b01b0e816318ea0"} Jul 02 21:44:07.287005 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.286979 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:07.381854 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.381826 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-bundle\") pod \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " Jul 02 21:44:07.382020 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.381880 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p66z9\" (UniqueName: \"kubernetes.io/projected/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-kube-api-access-p66z9\") pod \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " Jul 02 21:44:07.382020 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.381949 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-util\") pod \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\" (UID: \"da0515d7-8994-40c1-81b1-e9e15bbb1d5e\") " Jul 02 21:44:07.382611 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.382583 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-bundle" (OuterVolumeSpecName: "bundle") pod "da0515d7-8994-40c1-81b1-e9e15bbb1d5e" (UID: "da0515d7-8994-40c1-81b1-e9e15bbb1d5e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:44:07.383984 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.383963 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-kube-api-access-p66z9" (OuterVolumeSpecName: "kube-api-access-p66z9") pod "da0515d7-8994-40c1-81b1-e9e15bbb1d5e" (UID: "da0515d7-8994-40c1-81b1-e9e15bbb1d5e"). InnerVolumeSpecName "kube-api-access-p66z9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:44:07.387077 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.387051 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-util" (OuterVolumeSpecName: "util") pod "da0515d7-8994-40c1-81b1-e9e15bbb1d5e" (UID: "da0515d7-8994-40c1-81b1-e9e15bbb1d5e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:44:07.483168 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.483079 2547 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-util\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:44:07.483168 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.483109 2547 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:44:07.483168 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:07.483118 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p66z9\" (UniqueName: \"kubernetes.io/projected/da0515d7-8994-40c1-81b1-e9e15bbb1d5e-kube-api-access-p66z9\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:44:08.162538 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:08.162498 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" event={"ID":"da0515d7-8994-40c1-81b1-e9e15bbb1d5e","Type":"ContainerDied","Data":"d5e3d469b04d96f9b7f951d1d65ee044996f8a9e3afa9f3b9214261e68b510b6"} Jul 02 21:44:08.162538 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:08.162517 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmlvgn" Jul 02 21:44:08.162784 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:08.162550 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5e3d469b04d96f9b7f951d1d65ee044996f8a9e3afa9f3b9214261e68b510b6" Jul 02 21:44:17.557998 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.557960 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x"] Jul 02 21:44:17.558410 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.558286 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerName="pull" Jul 02 21:44:17.558410 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.558296 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerName="pull" Jul 02 21:44:17.558410 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.558314 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerName="extract" Jul 02 21:44:17.558410 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.558319 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerName="extract" Jul 02 21:44:17.558410 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.558332 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerName="util" Jul 02 21:44:17.558410 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.558338 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerName="util" Jul 02 21:44:17.558410 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.558400 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="da0515d7-8994-40c1-81b1-e9e15bbb1d5e" containerName="extract" Jul 02 21:44:17.561601 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.561580 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.567335 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.567312 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 21:44:17.567472 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.567446 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 21:44:17.567690 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.567675 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 21:44:17.568612 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.568595 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9ghtb\"" Jul 02 21:44:17.568612 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.568599 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 21:44:17.590347 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.590321 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x"] Jul 02 21:44:17.665477 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.665437 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85d72d11-b3d1-4c52-8556-bb51acdde7c5-manifests\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.665674 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.665483 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85d72d11-b3d1-4c52-8556-bb51acdde7c5-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.665674 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.665559 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85d72d11-b3d1-4c52-8556-bb51acdde7c5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.665674 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.665578 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rxw6\" (UniqueName: \"kubernetes.io/projected/85d72d11-b3d1-4c52-8556-bb51acdde7c5-kube-api-access-6rxw6\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.665674 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.665599 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85d72d11-b3d1-4c52-8556-bb51acdde7c5-tmp\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.766625 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.766586 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85d72d11-b3d1-4c52-8556-bb51acdde7c5-manifests\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.766625 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.766628 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85d72d11-b3d1-4c52-8556-bb51acdde7c5-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.766899 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.766677 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85d72d11-b3d1-4c52-8556-bb51acdde7c5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.766899 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.766693 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rxw6\" (UniqueName: \"kubernetes.io/projected/85d72d11-b3d1-4c52-8556-bb51acdde7c5-kube-api-access-6rxw6\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.766899 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.766713 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85d72d11-b3d1-4c52-8556-bb51acdde7c5-tmp\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.767085 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.767065 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/85d72d11-b3d1-4c52-8556-bb51acdde7c5-manifests\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.768995 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.768950 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85d72d11-b3d1-4c52-8556-bb51acdde7c5-tmp\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.769166 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.769114 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/85d72d11-b3d1-4c52-8556-bb51acdde7c5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.769218 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.769171 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/85d72d11-b3d1-4c52-8556-bb51acdde7c5-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.784894 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.784863 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rxw6\" (UniqueName: \"kubernetes.io/projected/85d72d11-b3d1-4c52-8556-bb51acdde7c5-kube-api-access-6rxw6\") pod \"opendatahub-operator-controller-manager-6c4697655-gpt2x\" (UID: \"85d72d11-b3d1-4c52-8556-bb51acdde7c5\") " pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:17.871812 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:17.871715 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:18.021802 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:18.021772 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x"] Jul 02 21:44:18.025106 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:44:18.025078 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d72d11_b3d1_4c52_8556_bb51acdde7c5.slice/crio-6e91d5a3357857c6315446f212456c99e0dc6975036a37a281dca3412f5c8ef1 WatchSource:0}: Error finding container 6e91d5a3357857c6315446f212456c99e0dc6975036a37a281dca3412f5c8ef1: Status 404 returned error can't find the container with id 6e91d5a3357857c6315446f212456c99e0dc6975036a37a281dca3412f5c8ef1 Jul 02 21:44:18.217026 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:18.216931 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" event={"ID":"85d72d11-b3d1-4c52-8556-bb51acdde7c5","Type":"ContainerStarted","Data":"6e91d5a3357857c6315446f212456c99e0dc6975036a37a281dca3412f5c8ef1"} Jul 02 21:44:21.236660 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:21.236622 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" event={"ID":"85d72d11-b3d1-4c52-8556-bb51acdde7c5","Type":"ContainerStarted","Data":"6160801fcfa68e5ec10092b54a7d32d3ca7842eb1ee72926bbe338002a8a0035"} Jul 02 21:44:22.242199 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:22.242162 2547 generic.go:356] "Generic (PLEG): container finished" podID="85d72d11-b3d1-4c52-8556-bb51acdde7c5" containerID="6160801fcfa68e5ec10092b54a7d32d3ca7842eb1ee72926bbe338002a8a0035" exitCode=0 Jul 02 21:44:22.242590 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:22.242254 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" event={"ID":"85d72d11-b3d1-4c52-8556-bb51acdde7c5","Type":"ContainerDied","Data":"6160801fcfa68e5ec10092b54a7d32d3ca7842eb1ee72926bbe338002a8a0035"} Jul 02 21:44:23.249362 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:23.249328 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" event={"ID":"85d72d11-b3d1-4c52-8556-bb51acdde7c5","Type":"ContainerStarted","Data":"f7d4dad0c601d17dde310423e5d32d616acc1eccdf6d7a1e9a6f1ee9a9dfdea3"} Jul 02 21:44:23.249735 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:23.249428 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:23.274889 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:23.274840 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" podStartSLOduration=1.808630166 podStartE2EDuration="6.274825517s" totalImagesPullingTime="3.386947184s" totalInitContainerRuntime="1.079248167s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:44:23.273740621 +0000 UTC m=+797.723579252" watchObservedRunningTime="2026-07-02 21:44:23.274825517 +0000 UTC m=+797.724664140" Jul 02 21:44:34.255866 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:34.255790 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c4697655-gpt2x" Jul 02 21:44:38.711725 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.711690 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f"] Jul 02 21:44:38.717153 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.717127 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.723436 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.723413 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nzzzz\"" Jul 02 21:44:38.723566 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.723539 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 21:44:38.723680 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.723656 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 21:44:38.735549 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.735529 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f"] Jul 02 21:44:38.742351 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.742327 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f9f105e-0883-4199-84ae-7c79c2f6f1dc-tmp\") pod \"kube-auth-proxy-6d45cbf66b-c2f2f\" (UID: \"4f9f105e-0883-4199-84ae-7c79c2f6f1dc\") " pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.742486 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.742364 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f9f105e-0883-4199-84ae-7c79c2f6f1dc-tls-certs\") pod \"kube-auth-proxy-6d45cbf66b-c2f2f\" (UID: \"4f9f105e-0883-4199-84ae-7c79c2f6f1dc\") " pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.742486 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.742444 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgx9z\" (UniqueName: \"kubernetes.io/projected/4f9f105e-0883-4199-84ae-7c79c2f6f1dc-kube-api-access-pgx9z\") pod \"kube-auth-proxy-6d45cbf66b-c2f2f\" (UID: \"4f9f105e-0883-4199-84ae-7c79c2f6f1dc\") " pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.843089 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.843049 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f9f105e-0883-4199-84ae-7c79c2f6f1dc-tmp\") pod \"kube-auth-proxy-6d45cbf66b-c2f2f\" (UID: \"4f9f105e-0883-4199-84ae-7c79c2f6f1dc\") " pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.843089 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.843094 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f9f105e-0883-4199-84ae-7c79c2f6f1dc-tls-certs\") pod \"kube-auth-proxy-6d45cbf66b-c2f2f\" (UID: \"4f9f105e-0883-4199-84ae-7c79c2f6f1dc\") " pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.843300 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.843156 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgx9z\" (UniqueName: \"kubernetes.io/projected/4f9f105e-0883-4199-84ae-7c79c2f6f1dc-kube-api-access-pgx9z\") pod \"kube-auth-proxy-6d45cbf66b-c2f2f\" (UID: \"4f9f105e-0883-4199-84ae-7c79c2f6f1dc\") " pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.845373 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.845340 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f9f105e-0883-4199-84ae-7c79c2f6f1dc-tmp\") pod \"kube-auth-proxy-6d45cbf66b-c2f2f\" (UID: \"4f9f105e-0883-4199-84ae-7c79c2f6f1dc\") " pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.845667 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.845646 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f9f105e-0883-4199-84ae-7c79c2f6f1dc-tls-certs\") pod \"kube-auth-proxy-6d45cbf66b-c2f2f\" (UID: \"4f9f105e-0883-4199-84ae-7c79c2f6f1dc\") " pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.866756 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.866730 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgx9z\" (UniqueName: \"kubernetes.io/projected/4f9f105e-0883-4199-84ae-7c79c2f6f1dc-kube-api-access-pgx9z\") pod \"kube-auth-proxy-6d45cbf66b-c2f2f\" (UID: \"4f9f105e-0883-4199-84ae-7c79c2f6f1dc\") " pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:38.899749 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.899713 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bqp65"] Jul 02 21:44:38.903763 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.903742 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:38.907124 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.907097 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hxwdw\"" Jul 02 21:44:38.907593 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.907574 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 02 21:44:38.924759 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.924728 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bqp65"] Jul 02 21:44:38.944122 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.944093 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xmdm\" (UniqueName: \"kubernetes.io/projected/49cca079-af7a-44a6-b044-574a00ec939f-kube-api-access-8xmdm\") pod \"odh-model-controller-7998c67dc4-bqp65\" (UID: \"49cca079-af7a-44a6-b044-574a00ec939f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:38.944280 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:38.944143 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49cca079-af7a-44a6-b044-574a00ec939f-cert\") pod \"odh-model-controller-7998c67dc4-bqp65\" (UID: \"49cca079-af7a-44a6-b044-574a00ec939f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:39.028336 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.028250 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" Jul 02 21:44:39.045210 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.045174 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xmdm\" (UniqueName: \"kubernetes.io/projected/49cca079-af7a-44a6-b044-574a00ec939f-kube-api-access-8xmdm\") pod \"odh-model-controller-7998c67dc4-bqp65\" (UID: \"49cca079-af7a-44a6-b044-574a00ec939f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:39.045344 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.045266 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49cca079-af7a-44a6-b044-574a00ec939f-cert\") pod \"odh-model-controller-7998c67dc4-bqp65\" (UID: \"49cca079-af7a-44a6-b044-574a00ec939f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:39.045456 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:44:39.045430 2547 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 02 21:44:39.045531 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:44:39.045504 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49cca079-af7a-44a6-b044-574a00ec939f-cert podName:49cca079-af7a-44a6-b044-574a00ec939f nodeName:}" failed. No retries permitted until 2026-07-02 21:44:39.545483844 +0000 UTC m=+813.995322462 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/49cca079-af7a-44a6-b044-574a00ec939f-cert") pod "odh-model-controller-7998c67dc4-bqp65" (UID: "49cca079-af7a-44a6-b044-574a00ec939f") : secret "odh-model-controller-webhook-cert" not found Jul 02 21:44:39.062437 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.062407 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xmdm\" (UniqueName: \"kubernetes.io/projected/49cca079-af7a-44a6-b044-574a00ec939f-kube-api-access-8xmdm\") pod \"odh-model-controller-7998c67dc4-bqp65\" (UID: \"49cca079-af7a-44a6-b044-574a00ec939f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:39.173038 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.172989 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f"] Jul 02 21:44:39.176855 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:44:39.176827 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f9f105e_0883_4199_84ae_7c79c2f6f1dc.slice/crio-bb2731d6b519a4685e7e8ccf2898e93615d18e80e3beb103157d5270c1a21aa5 WatchSource:0}: Error finding container bb2731d6b519a4685e7e8ccf2898e93615d18e80e3beb103157d5270c1a21aa5: Status 404 returned error can't find the container with id bb2731d6b519a4685e7e8ccf2898e93615d18e80e3beb103157d5270c1a21aa5 Jul 02 21:44:39.325156 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.325076 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" event={"ID":"4f9f105e-0883-4199-84ae-7c79c2f6f1dc","Type":"ContainerStarted","Data":"bb2731d6b519a4685e7e8ccf2898e93615d18e80e3beb103157d5270c1a21aa5"} Jul 02 21:44:39.550311 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.550278 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49cca079-af7a-44a6-b044-574a00ec939f-cert\") pod \"odh-model-controller-7998c67dc4-bqp65\" (UID: \"49cca079-af7a-44a6-b044-574a00ec939f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:39.552860 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.552832 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49cca079-af7a-44a6-b044-574a00ec939f-cert\") pod \"odh-model-controller-7998c67dc4-bqp65\" (UID: \"49cca079-af7a-44a6-b044-574a00ec939f\") " pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:39.809567 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.809525 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-t577c"] Jul 02 21:44:39.813617 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.813589 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:39.815108 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.815089 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:39.816392 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.816362 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 02 21:44:39.831140 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.830788 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-t577c"] Jul 02 21:44:39.852759 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.852694 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsb5x\" (UniqueName: \"kubernetes.io/projected/02d4eac5-d07a-4f24-893a-b24c5cec4b1d-kube-api-access-gsb5x\") pod \"model-serving-api-54d89bd5fc-t577c\" (UID: \"02d4eac5-d07a-4f24-893a-b24c5cec4b1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:39.852928 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.852779 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02d4eac5-d07a-4f24-893a-b24c5cec4b1d-tls-certs\") pod \"model-serving-api-54d89bd5fc-t577c\" (UID: \"02d4eac5-d07a-4f24-893a-b24c5cec4b1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:39.954365 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.954331 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsb5x\" (UniqueName: \"kubernetes.io/projected/02d4eac5-d07a-4f24-893a-b24c5cec4b1d-kube-api-access-gsb5x\") pod \"model-serving-api-54d89bd5fc-t577c\" (UID: \"02d4eac5-d07a-4f24-893a-b24c5cec4b1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:39.954549 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.954407 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02d4eac5-d07a-4f24-893a-b24c5cec4b1d-tls-certs\") pod \"model-serving-api-54d89bd5fc-t577c\" (UID: \"02d4eac5-d07a-4f24-893a-b24c5cec4b1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:39.954549 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:44:39.954513 2547 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 02 21:44:39.954661 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:44:39.954570 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d4eac5-d07a-4f24-893a-b24c5cec4b1d-tls-certs podName:02d4eac5-d07a-4f24-893a-b24c5cec4b1d nodeName:}" failed. No retries permitted until 2026-07-02 21:44:40.454554136 +0000 UTC m=+814.904392759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/02d4eac5-d07a-4f24-893a-b24c5cec4b1d-tls-certs") pod "model-serving-api-54d89bd5fc-t577c" (UID: "02d4eac5-d07a-4f24-893a-b24c5cec4b1d") : secret "model-serving-api-tls" not found Jul 02 21:44:39.963349 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.963309 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsb5x\" (UniqueName: \"kubernetes.io/projected/02d4eac5-d07a-4f24-893a-b24c5cec4b1d-kube-api-access-gsb5x\") pod \"model-serving-api-54d89bd5fc-t577c\" (UID: \"02d4eac5-d07a-4f24-893a-b24c5cec4b1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:39.979339 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:39.979319 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bqp65"] Jul 02 21:44:39.982866 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:44:39.982834 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49cca079_af7a_44a6_b044_574a00ec939f.slice/crio-e0514603fd80513b1c679419cb12268d73c9e87bb24c6e8bf4a4d9b12d6a5498 WatchSource:0}: Error finding container e0514603fd80513b1c679419cb12268d73c9e87bb24c6e8bf4a4d9b12d6a5498: Status 404 returned error can't find the container with id e0514603fd80513b1c679419cb12268d73c9e87bb24c6e8bf4a4d9b12d6a5498 Jul 02 21:44:40.108292 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.108205 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t"] Jul 02 21:44:40.113725 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.113701 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.116726 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.116688 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 21:44:40.117281 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.117253 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pkmfn\"" Jul 02 21:44:40.117396 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.117267 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 21:44:40.117761 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.117745 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 21:44:40.117858 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.117752 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 21:44:40.128522 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.128500 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t"] Jul 02 21:44:40.162278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.161788 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-token\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.162278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.161889 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.162278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.161923 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-cacerts\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.162278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.162003 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.162278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.162089 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4720a4-8a6f-490d-bad5-4c402141b6e3-local-certs\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.162278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.162115 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqbl8\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-kube-api-access-sqbl8\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.162278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.162184 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.262934 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.262894 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.262934 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.262932 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-token\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.263193 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.262971 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.263193 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.262997 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-cacerts\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.263193 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.263123 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.263353 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.263195 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4720a4-8a6f-490d-bad5-4c402141b6e3-local-certs\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.263353 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.263221 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqbl8\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-kube-api-access-sqbl8\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.263938 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.263905 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.265889 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.265866 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-cacerts\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.266591 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.266539 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.266694 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.266570 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4720a4-8a6f-490d-bad5-4c402141b6e3-local-certs\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.266694 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.266658 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.272932 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.272883 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-token\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.273410 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.273368 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqbl8\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-kube-api-access-sqbl8\") pod \"istiod-openshift-gateway-7574459656-w6p9t\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.331936 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.331903 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" event={"ID":"49cca079-af7a-44a6-b044-574a00ec939f","Type":"ContainerStarted","Data":"e0514603fd80513b1c679419cb12268d73c9e87bb24c6e8bf4a4d9b12d6a5498"} Jul 02 21:44:40.426175 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.426092 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:40.465148 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.465016 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02d4eac5-d07a-4f24-893a-b24c5cec4b1d-tls-certs\") pod \"model-serving-api-54d89bd5fc-t577c\" (UID: \"02d4eac5-d07a-4f24-893a-b24c5cec4b1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:40.468701 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.468666 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02d4eac5-d07a-4f24-893a-b24c5cec4b1d-tls-certs\") pod \"model-serving-api-54d89bd5fc-t577c\" (UID: \"02d4eac5-d07a-4f24-893a-b24c5cec4b1d\") " pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:40.602955 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.602918 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t"] Jul 02 21:44:40.606819 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:44:40.606782 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d4720a4_8a6f_490d_bad5_4c402141b6e3.slice/crio-28d897e77bc342b471975c9c7c738c95fb1b3417bbfcfc3070f0dc5a1513e82e WatchSource:0}: Error finding container 28d897e77bc342b471975c9c7c738c95fb1b3417bbfcfc3070f0dc5a1513e82e: Status 404 returned error can't find the container with id 28d897e77bc342b471975c9c7c738c95fb1b3417bbfcfc3070f0dc5a1513e82e Jul 02 21:44:40.727834 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.727641 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:40.908058 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:40.908023 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-t577c"] Jul 02 21:44:41.347178 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:41.347091 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" event={"ID":"2d4720a4-8a6f-490d-bad5-4c402141b6e3","Type":"ContainerStarted","Data":"28d897e77bc342b471975c9c7c738c95fb1b3417bbfcfc3070f0dc5a1513e82e"} Jul 02 21:44:41.348858 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:41.348823 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-t577c" event={"ID":"02d4eac5-d07a-4f24-893a-b24c5cec4b1d","Type":"ContainerStarted","Data":"dd4281bd99835abf3ab2a18e96053862ec08d39275db14acf1d04021bc78e7e5"} Jul 02 21:44:44.366685 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:44.366649 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" event={"ID":"4f9f105e-0883-4199-84ae-7c79c2f6f1dc","Type":"ContainerStarted","Data":"7a34a7b1c2f24cf383f9351fe6db5ceb2bfb67215b837abcdf14f53d812c9020"} Jul 02 21:44:44.368658 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:44.368624 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-t577c" event={"ID":"02d4eac5-d07a-4f24-893a-b24c5cec4b1d","Type":"ContainerStarted","Data":"39a08c4f3bfb490018268809d5a9566947227b2c81d791a0b4dccd853bd2f0e8"} Jul 02 21:44:44.368789 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:44.368696 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:44.370252 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:44.370224 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" event={"ID":"49cca079-af7a-44a6-b044-574a00ec939f","Type":"ContainerStarted","Data":"ff5eed2ac4f985d27cb61a8b11701a267293eb56cc157d1b1875f5ef3c4f3452"} Jul 02 21:44:44.370409 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:44.370390 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:44.383752 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:44.383715 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6d45cbf66b-c2f2f" podStartSLOduration=2.11768427 podStartE2EDuration="6.383697343s" totalImagesPullingTime="4.266013073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:44:44.382930839 +0000 UTC m=+818.832769464" watchObservedRunningTime="2026-07-02 21:44:44.383697343 +0000 UTC m=+818.833535968" Jul 02 21:44:44.402220 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:44.402171 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" podStartSLOduration=2.891471385 podStartE2EDuration="6.402153284s" totalImagesPullingTime="3.510681899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:44:44.401770051 +0000 UTC m=+818.851608676" watchObservedRunningTime="2026-07-02 21:44:44.402153284 +0000 UTC m=+818.851991910" Jul 02 21:44:44.421248 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:44.421193 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-t577c" podStartSLOduration=2.30261638 podStartE2EDuration="5.421171544s" totalImagesPullingTime="3.118555164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:44:44.419560939 +0000 UTC m=+818.869399563" watchObservedRunningTime="2026-07-02 21:44:44.421171544 +0000 UTC m=+818.871010169" Jul 02 21:44:50.372374 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.372336 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb"] Jul 02 21:44:50.376364 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.376343 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:50.396212 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.396189 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-z5vl7\"" Jul 02 21:44:50.396341 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.396292 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 02 21:44:50.398652 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.398631 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm"] Jul 02 21:44:50.401871 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.401853 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb"] Jul 02 21:44:50.401983 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.401967 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:44:50.406211 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.406188 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d8g7r\"" Jul 02 21:44:50.407056 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.407040 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 02 21:44:50.454581 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.454552 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm"] Jul 02 21:44:50.462334 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.462311 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp9nf\" (UniqueName: \"kubernetes.io/projected/69be0910-f104-433e-a97d-43ab24d5cd82-kube-api-access-xp9nf\") pod \"kserve-controller-manager-d6f4b65b8-jxqlb\" (UID: \"69be0910-f104-433e-a97d-43ab24d5cd82\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:50.462490 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.462345 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69be0910-f104-433e-a97d-43ab24d5cd82-cert\") pod \"kserve-controller-manager-d6f4b65b8-jxqlb\" (UID: \"69be0910-f104-433e-a97d-43ab24d5cd82\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:50.462490 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.462389 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdrt8\" (UniqueName: \"kubernetes.io/projected/ced64b90-2bad-4247-afd2-f4776cb690fd-kube-api-access-vdrt8\") pod \"llmisvc-controller-manager-777c6bd7d5-7vbvm\" (UID: \"ced64b90-2bad-4247-afd2-f4776cb690fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:44:50.462490 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.462482 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ced64b90-2bad-4247-afd2-f4776cb690fd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7vbvm\" (UID: \"ced64b90-2bad-4247-afd2-f4776cb690fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:44:50.562964 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.562931 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp9nf\" (UniqueName: \"kubernetes.io/projected/69be0910-f104-433e-a97d-43ab24d5cd82-kube-api-access-xp9nf\") pod \"kserve-controller-manager-d6f4b65b8-jxqlb\" (UID: \"69be0910-f104-433e-a97d-43ab24d5cd82\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:50.562964 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.562970 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69be0910-f104-433e-a97d-43ab24d5cd82-cert\") pod \"kserve-controller-manager-d6f4b65b8-jxqlb\" (UID: \"69be0910-f104-433e-a97d-43ab24d5cd82\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:50.563234 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.562999 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdrt8\" (UniqueName: \"kubernetes.io/projected/ced64b90-2bad-4247-afd2-f4776cb690fd-kube-api-access-vdrt8\") pod \"llmisvc-controller-manager-777c6bd7d5-7vbvm\" (UID: \"ced64b90-2bad-4247-afd2-f4776cb690fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:44:50.563234 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.563018 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ced64b90-2bad-4247-afd2-f4776cb690fd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7vbvm\" (UID: \"ced64b90-2bad-4247-afd2-f4776cb690fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:44:50.563234 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:44:50.563110 2547 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 02 21:44:50.563234 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:44:50.563192 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69be0910-f104-433e-a97d-43ab24d5cd82-cert podName:69be0910-f104-433e-a97d-43ab24d5cd82 nodeName:}" failed. No retries permitted until 2026-07-02 21:44:51.063171194 +0000 UTC m=+825.513009796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/69be0910-f104-433e-a97d-43ab24d5cd82-cert") pod "kserve-controller-manager-d6f4b65b8-jxqlb" (UID: "69be0910-f104-433e-a97d-43ab24d5cd82") : secret "kserve-webhook-server-cert" not found Jul 02 21:44:50.565450 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.565434 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ced64b90-2bad-4247-afd2-f4776cb690fd-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-7vbvm\" (UID: \"ced64b90-2bad-4247-afd2-f4776cb690fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:44:50.581093 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.581068 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp9nf\" (UniqueName: \"kubernetes.io/projected/69be0910-f104-433e-a97d-43ab24d5cd82-kube-api-access-xp9nf\") pod \"kserve-controller-manager-d6f4b65b8-jxqlb\" (UID: \"69be0910-f104-433e-a97d-43ab24d5cd82\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:50.586087 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.586065 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdrt8\" (UniqueName: \"kubernetes.io/projected/ced64b90-2bad-4247-afd2-f4776cb690fd-kube-api-access-vdrt8\") pod \"llmisvc-controller-manager-777c6bd7d5-7vbvm\" (UID: \"ced64b90-2bad-4247-afd2-f4776cb690fd\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:44:50.711958 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.711863 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:44:50.846822 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:50.846793 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm"] Jul 02 21:44:50.848485 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:44:50.848453 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podced64b90_2bad_4247_afd2_f4776cb690fd.slice/crio-ade039088529e90b24c292d12b1cb1057c71603dbba12f97b270c9b4b2380ddd WatchSource:0}: Error finding container ade039088529e90b24c292d12b1cb1057c71603dbba12f97b270c9b4b2380ddd: Status 404 returned error can't find the container with id ade039088529e90b24c292d12b1cb1057c71603dbba12f97b270c9b4b2380ddd Jul 02 21:44:51.067578 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:51.067537 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69be0910-f104-433e-a97d-43ab24d5cd82-cert\") pod \"kserve-controller-manager-d6f4b65b8-jxqlb\" (UID: \"69be0910-f104-433e-a97d-43ab24d5cd82\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:51.067748 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:44:51.067663 2547 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 02 21:44:51.067748 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:44:51.067719 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69be0910-f104-433e-a97d-43ab24d5cd82-cert podName:69be0910-f104-433e-a97d-43ab24d5cd82 nodeName:}" failed. No retries permitted until 2026-07-02 21:44:52.06770346 +0000 UTC m=+826.517542063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/69be0910-f104-433e-a97d-43ab24d5cd82-cert") pod "kserve-controller-manager-d6f4b65b8-jxqlb" (UID: "69be0910-f104-433e-a97d-43ab24d5cd82") : secret "kserve-webhook-server-cert" not found Jul 02 21:44:51.413824 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:51.413744 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" event={"ID":"ced64b90-2bad-4247-afd2-f4776cb690fd","Type":"ContainerStarted","Data":"ade039088529e90b24c292d12b1cb1057c71603dbba12f97b270c9b4b2380ddd"} Jul 02 21:44:52.078436 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:52.078399 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69be0910-f104-433e-a97d-43ab24d5cd82-cert\") pod \"kserve-controller-manager-d6f4b65b8-jxqlb\" (UID: \"69be0910-f104-433e-a97d-43ab24d5cd82\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:52.081298 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:52.081275 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/69be0910-f104-433e-a97d-43ab24d5cd82-cert\") pod \"kserve-controller-manager-d6f4b65b8-jxqlb\" (UID: \"69be0910-f104-433e-a97d-43ab24d5cd82\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:52.187000 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:52.186839 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:53.753836 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:53.753790 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:44:53.754197 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:53.753892 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:44:53.946267 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:53.946231 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb"] Jul 02 21:44:54.440883 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:44:54.440848 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69be0910_f104_433e_a97d_43ab24d5cd82.slice/crio-b5886eeea02909dcecb37c6f4a14234d6b426f52ef219975b65b6e4bc9e3d44f WatchSource:0}: Error finding container b5886eeea02909dcecb37c6f4a14234d6b426f52ef219975b65b6e4bc9e3d44f: Status 404 returned error can't find the container with id b5886eeea02909dcecb37c6f4a14234d6b426f52ef219975b65b6e4bc9e3d44f Jul 02 21:44:55.380066 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.379964 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bqp65" Jul 02 21:44:55.381588 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.381556 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-t577c" Jul 02 21:44:55.441398 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.441341 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" event={"ID":"69be0910-f104-433e-a97d-43ab24d5cd82","Type":"ContainerStarted","Data":"b5886eeea02909dcecb37c6f4a14234d6b426f52ef219975b65b6e4bc9e3d44f"} Jul 02 21:44:55.443904 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.443633 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" event={"ID":"2d4720a4-8a6f-490d-bad5-4c402141b6e3","Type":"ContainerStarted","Data":"24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7"} Jul 02 21:44:55.443904 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.443771 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:55.445304 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.445281 2547 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-w6p9t container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 21:44:55.445448 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.445322 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" podUID="2d4720a4-8a6f-490d-bad5-4c402141b6e3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 21:44:55.445891 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.445871 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" event={"ID":"ced64b90-2bad-4247-afd2-f4776cb690fd","Type":"ContainerStarted","Data":"3a7c91814aa5a8228df3ee1834c6b222dcac95c55da34dd0722094a19dd93ff8"} Jul 02 21:44:55.446023 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.446006 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:44:55.528399 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.527798 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" podStartSLOduration=2.3839449249999998 podStartE2EDuration="15.527776068s" totalImagesPullingTime="13.143831143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:44:55.52581558 +0000 UTC m=+829.975654208" watchObservedRunningTime="2026-07-02 21:44:55.527776068 +0000 UTC m=+829.977614693" Jul 02 21:44:55.567529 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.566170 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" podStartSLOduration=1.922689584 podStartE2EDuration="5.566115923s" totalImagesPullingTime="3.643426339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:44:55.563860098 +0000 UTC m=+830.013698722" watchObservedRunningTime="2026-07-02 21:44:55.566115923 +0000 UTC m=+830.015954551" Jul 02 21:44:55.761156 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.761063 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7"] Jul 02 21:44:55.764735 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.764711 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:55.770193 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.770170 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kfcxd\"" Jul 02 21:44:55.793054 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.793021 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7"] Jul 02 21:44:55.921556 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.921518 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrt9v\" (UniqueName: \"kubernetes.io/projected/b3624094-3709-40d5-b87c-21645a759e14-kube-api-access-nrt9v\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:55.921717 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.921599 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:55.921717 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.921650 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:55.921717 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.921688 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:55.921878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.921719 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3624094-3709-40d5-b87c-21645a759e14-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:55.921878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.921799 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:55.921945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.921878 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:55.921945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.921922 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3624094-3709-40d5-b87c-21645a759e14-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:55.922036 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:55.921986 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3624094-3709-40d5-b87c-21645a759e14-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.022512 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022430 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.022512 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022483 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.022747 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022514 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3624094-3709-40d5-b87c-21645a759e14-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.022747 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022542 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3624094-3709-40d5-b87c-21645a759e14-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.022747 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022563 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrt9v\" (UniqueName: \"kubernetes.io/projected/b3624094-3709-40d5-b87c-21645a759e14-kube-api-access-nrt9v\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.022747 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022588 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.022747 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022613 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.022747 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022663 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.022747 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022697 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3624094-3709-40d5-b87c-21645a759e14-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.023098 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.022883 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.023193 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.023170 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.023254 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.023214 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.023254 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.023227 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.023355 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.023314 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b3624094-3709-40d5-b87c-21645a759e14-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.024872 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.024845 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b3624094-3709-40d5-b87c-21645a759e14-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.025016 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.024989 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b3624094-3709-40d5-b87c-21645a759e14-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.041994 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.041965 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrt9v\" (UniqueName: \"kubernetes.io/projected/b3624094-3709-40d5-b87c-21645a759e14-kube-api-access-nrt9v\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.042990 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.042969 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3624094-3709-40d5-b87c-21645a759e14-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4jjpt7\" (UID: \"b3624094-3709-40d5-b87c-21645a759e14\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.076183 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.076131 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:44:56.264855 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.264823 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7"] Jul 02 21:44:56.453153 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:56.453117 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:44:56.806639 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:44:56.806603 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3624094_3709_40d5_b87c_21645a759e14.slice/crio-98c8e372483a31651864bee7c70161a1ba8244499178c6186824e910880c53d9 WatchSource:0}: Error finding container 98c8e372483a31651864bee7c70161a1ba8244499178c6186824e910880c53d9: Status 404 returned error can't find the container with id 98c8e372483a31651864bee7c70161a1ba8244499178c6186824e910880c53d9 Jul 02 21:44:57.460078 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:57.460000 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" event={"ID":"69be0910-f104-433e-a97d-43ab24d5cd82","Type":"ContainerStarted","Data":"a34dfb62e43a9173a2e3f91c493fdf9af8658aa2b44e42fc7cb44d58dbcb004e"} Jul 02 21:44:57.460658 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:57.460257 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:44:57.461587 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:57.461561 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" event={"ID":"b3624094-3709-40d5-b87c-21645a759e14","Type":"ContainerStarted","Data":"98c8e372483a31651864bee7c70161a1ba8244499178c6186824e910880c53d9"} Jul 02 21:44:57.497659 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:57.497566 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" podStartSLOduration=5.083355681 podStartE2EDuration="7.497543853s" totalImagesPullingTime="2.414188172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:44:57.496239587 +0000 UTC m=+831.946078211" watchObservedRunningTime="2026-07-02 21:44:57.497543853 +0000 UTC m=+831.947382479" Jul 02 21:44:59.519354 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:59.519316 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:44:59.519731 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:59.519408 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:44:59.519731 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:44:59.519440 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:45:00.478888 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:00.478838 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" event={"ID":"b3624094-3709-40d5-b87c-21645a759e14","Type":"ContainerStarted","Data":"33c2e29d0ca0bf4c0eae7315b038f2beceeb4cb760cac7b90a1ebb4bbbee6dce"} Jul 02 21:45:00.502793 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:00.502739 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" podStartSLOduration=2.792582258 podStartE2EDuration="5.502722498s" totalImagesPullingTime="2.71014024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:44:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:45:00.501596542 +0000 UTC m=+834.951435167" watchObservedRunningTime="2026-07-02 21:45:00.502722498 +0000 UTC m=+834.952561122" Jul 02 21:45:01.076602 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:01.076560 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:45:01.081289 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:01.081260 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:45:01.485962 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:01.485883 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:45:01.487050 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:01.487023 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4jjpt7" Jul 02 21:45:15.624875 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:15.624839 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t"] Jul 02 21:45:15.625501 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:15.625093 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" podUID="2d4720a4-8a6f-490d-bad5-4c402141b6e3" containerName="discovery" containerID="cri-o://24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7" gracePeriod=30 Jul 02 21:45:15.901192 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:15.901125 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:45:16.008616 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.008588 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4720a4-8a6f-490d-bad5-4c402141b6e3-local-certs\") pod \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " Jul 02 21:45:16.008806 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.008631 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-dns-cert\") pod \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " Jul 02 21:45:16.008806 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.008666 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-kubeconfig\") pod \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " Jul 02 21:45:16.008806 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.008690 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-token\") pod \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " Jul 02 21:45:16.008806 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.008712 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-cacerts\") pod \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " Jul 02 21:45:16.008806 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.008745 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-ca-configmap\") pod \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " Jul 02 21:45:16.009084 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.008820 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqbl8\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-kube-api-access-sqbl8\") pod \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\" (UID: \"2d4720a4-8a6f-490d-bad5-4c402141b6e3\") " Jul 02 21:45:16.009414 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.009365 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "2d4720a4-8a6f-490d-bad5-4c402141b6e3" (UID: "2d4720a4-8a6f-490d-bad5-4c402141b6e3"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:45:16.011217 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.011181 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "2d4720a4-8a6f-490d-bad5-4c402141b6e3" (UID: "2d4720a4-8a6f-490d-bad5-4c402141b6e3"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:45:16.011367 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.011195 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "2d4720a4-8a6f-490d-bad5-4c402141b6e3" (UID: "2d4720a4-8a6f-490d-bad5-4c402141b6e3"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:45:16.011367 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.011194 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d4720a4-8a6f-490d-bad5-4c402141b6e3-local-certs" (OuterVolumeSpecName: "local-certs") pod "2d4720a4-8a6f-490d-bad5-4c402141b6e3" (UID: "2d4720a4-8a6f-490d-bad5-4c402141b6e3"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:45:16.011367 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.011352 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-kube-api-access-sqbl8" (OuterVolumeSpecName: "kube-api-access-sqbl8") pod "2d4720a4-8a6f-490d-bad5-4c402141b6e3" (UID: "2d4720a4-8a6f-490d-bad5-4c402141b6e3"). InnerVolumeSpecName "kube-api-access-sqbl8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:45:16.011540 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.011372 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d4720a4-8a6f-490d-bad5-4c402141b6e3" (UID: "2d4720a4-8a6f-490d-bad5-4c402141b6e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:45:16.011577 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.011560 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-cacerts" (OuterVolumeSpecName: "cacerts") pod "2d4720a4-8a6f-490d-bad5-4c402141b6e3" (UID: "2d4720a4-8a6f-490d-bad5-4c402141b6e3"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:45:16.109835 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.109808 2547 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-dns-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:16.109835 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.109832 2547 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-kubeconfig\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:16.110024 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.109844 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:16.110024 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.109855 2547 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2d4720a4-8a6f-490d-bad5-4c402141b6e3-cacerts\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:16.110024 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.109863 2547 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2d4720a4-8a6f-490d-bad5-4c402141b6e3-istio-csr-ca-configmap\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:16.110024 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.109872 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqbl8\" (UniqueName: \"kubernetes.io/projected/2d4720a4-8a6f-490d-bad5-4c402141b6e3-kube-api-access-sqbl8\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:16.110024 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.109881 2547 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d4720a4-8a6f-490d-bad5-4c402141b6e3-local-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:16.560438 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.560405 2547 generic.go:356] "Generic (PLEG): container finished" podID="2d4720a4-8a6f-490d-bad5-4c402141b6e3" containerID="24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7" exitCode=0 Jul 02 21:45:16.560438 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.560442 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" Jul 02 21:45:16.560752 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.560456 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" event={"ID":"2d4720a4-8a6f-490d-bad5-4c402141b6e3","Type":"ContainerDied","Data":"24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7"} Jul 02 21:45:16.560752 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.560502 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t" event={"ID":"2d4720a4-8a6f-490d-bad5-4c402141b6e3","Type":"ContainerDied","Data":"28d897e77bc342b471975c9c7c738c95fb1b3417bbfcfc3070f0dc5a1513e82e"} Jul 02 21:45:16.560752 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.560524 2547 scope.go:117] "RemoveContainer" containerID="24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7" Jul 02 21:45:16.569889 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.569874 2547 scope.go:117] "RemoveContainer" containerID="24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7" Jul 02 21:45:16.570127 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:45:16.570110 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7\": container with ID starting with 24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7 not found: ID does not exist" containerID="24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7" Jul 02 21:45:16.570171 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.570135 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7"} err="failed to get container status \"24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7\": rpc error: code = NotFound desc = could not find container \"24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7\": container with ID starting with 24cbcea6cdf642a1a562da2c07440e6fa00002d46c373c769ed5fd3a20dfc5d7 not found: ID does not exist" Jul 02 21:45:16.583405 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.583359 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t"] Jul 02 21:45:16.586296 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:16.586271 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-w6p9t"] Jul 02 21:45:18.007977 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:18.007942 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d4720a4-8a6f-490d-bad5-4c402141b6e3" path="/var/lib/kubelet/pods/2d4720a4-8a6f-490d-bad5-4c402141b6e3/volumes" Jul 02 21:45:26.454859 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:26.454827 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-7vbvm" Jul 02 21:45:28.473010 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:28.472977 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-jxqlb" Jul 02 21:45:39.655333 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.655298 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dxrpc"] Jul 02 21:45:39.655785 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.655715 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d4720a4-8a6f-490d-bad5-4c402141b6e3" containerName="discovery" Jul 02 21:45:39.655785 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.655730 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d4720a4-8a6f-490d-bad5-4c402141b6e3" containerName="discovery" Jul 02 21:45:39.655888 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.655816 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d4720a4-8a6f-490d-bad5-4c402141b6e3" containerName="discovery" Jul 02 21:45:39.658908 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.658889 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" Jul 02 21:45:39.662510 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.662488 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-vh66x\"" Jul 02 21:45:39.662645 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.662539 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 21:45:39.662905 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.662889 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 21:45:39.668417 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.668372 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dxrpc"] Jul 02 21:45:39.820886 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.820855 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtb4z\" (UniqueName: \"kubernetes.io/projected/ba75e276-953f-46d2-8ad8-510f8896b6c4-kube-api-access-vtb4z\") pod \"kuadrant-operator-catalog-dxrpc\" (UID: \"ba75e276-953f-46d2-8ad8-510f8896b6c4\") " pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" Jul 02 21:45:39.922050 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.921961 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtb4z\" (UniqueName: \"kubernetes.io/projected/ba75e276-953f-46d2-8ad8-510f8896b6c4-kube-api-access-vtb4z\") pod \"kuadrant-operator-catalog-dxrpc\" (UID: \"ba75e276-953f-46d2-8ad8-510f8896b6c4\") " pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" Jul 02 21:45:39.937518 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.937486 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtb4z\" (UniqueName: \"kubernetes.io/projected/ba75e276-953f-46d2-8ad8-510f8896b6c4-kube-api-access-vtb4z\") pod \"kuadrant-operator-catalog-dxrpc\" (UID: \"ba75e276-953f-46d2-8ad8-510f8896b6c4\") " pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" Jul 02 21:45:39.970492 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.970460 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" Jul 02 21:45:39.987910 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:39.987882 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dxrpc"] Jul 02 21:45:40.124611 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.124588 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dxrpc"] Jul 02 21:45:40.127245 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:45:40.127215 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba75e276_953f_46d2_8ad8_510f8896b6c4.slice/crio-650e44b46bf58b92df0240b5efdb1557c361f5783e702b0f106395e59b38f3af WatchSource:0}: Error finding container 650e44b46bf58b92df0240b5efdb1557c361f5783e702b0f106395e59b38f3af: Status 404 returned error can't find the container with id 650e44b46bf58b92df0240b5efdb1557c361f5783e702b0f106395e59b38f3af Jul 02 21:45:40.196944 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.196862 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h52vw"] Jul 02 21:45:40.201616 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.201596 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h52vw" Jul 02 21:45:40.207997 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.207975 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h52vw"] Jul 02 21:45:40.325857 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.325824 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs9nc\" (UniqueName: \"kubernetes.io/projected/62cc0b04-447b-4445-a978-b3e40f5a3296-kube-api-access-rs9nc\") pod \"kuadrant-operator-catalog-h52vw\" (UID: \"62cc0b04-447b-4445-a978-b3e40f5a3296\") " pod="kuadrant-system/kuadrant-operator-catalog-h52vw" Jul 02 21:45:40.426834 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.426795 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs9nc\" (UniqueName: \"kubernetes.io/projected/62cc0b04-447b-4445-a978-b3e40f5a3296-kube-api-access-rs9nc\") pod \"kuadrant-operator-catalog-h52vw\" (UID: \"62cc0b04-447b-4445-a978-b3e40f5a3296\") " pod="kuadrant-system/kuadrant-operator-catalog-h52vw" Jul 02 21:45:40.435855 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.435826 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs9nc\" (UniqueName: \"kubernetes.io/projected/62cc0b04-447b-4445-a978-b3e40f5a3296-kube-api-access-rs9nc\") pod \"kuadrant-operator-catalog-h52vw\" (UID: \"62cc0b04-447b-4445-a978-b3e40f5a3296\") " pod="kuadrant-system/kuadrant-operator-catalog-h52vw" Jul 02 21:45:40.512134 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.512031 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-h52vw" Jul 02 21:45:40.651708 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.651682 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-h52vw"] Jul 02 21:45:40.653482 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:45:40.653455 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62cc0b04_447b_4445_a978_b3e40f5a3296.slice/crio-2afc08aa1bc7ef479dfeb5f3ea9b0e2b0ec6dc8ab339d910f75c478a84b01e5e WatchSource:0}: Error finding container 2afc08aa1bc7ef479dfeb5f3ea9b0e2b0ec6dc8ab339d910f75c478a84b01e5e: Status 404 returned error can't find the container with id 2afc08aa1bc7ef479dfeb5f3ea9b0e2b0ec6dc8ab339d910f75c478a84b01e5e Jul 02 21:45:40.692896 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.692864 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h52vw" event={"ID":"62cc0b04-447b-4445-a978-b3e40f5a3296","Type":"ContainerStarted","Data":"2afc08aa1bc7ef479dfeb5f3ea9b0e2b0ec6dc8ab339d910f75c478a84b01e5e"} Jul 02 21:45:40.694072 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:40.694045 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" event={"ID":"ba75e276-953f-46d2-8ad8-510f8896b6c4","Type":"ContainerStarted","Data":"650e44b46bf58b92df0240b5efdb1557c361f5783e702b0f106395e59b38f3af"} Jul 02 21:45:42.709714 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:42.709618 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-h52vw" event={"ID":"62cc0b04-447b-4445-a978-b3e40f5a3296","Type":"ContainerStarted","Data":"b3706e3f697c925ac0a9f9fa1292e8ad4a51223a6de141ff52fe8f6ad1f0f200"} Jul 02 21:45:42.711174 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:42.711151 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" event={"ID":"ba75e276-953f-46d2-8ad8-510f8896b6c4","Type":"ContainerStarted","Data":"4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810"} Jul 02 21:45:42.711287 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:42.711239 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" podUID="ba75e276-953f-46d2-8ad8-510f8896b6c4" containerName="registry-server" containerID="cri-o://4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810" gracePeriod=2 Jul 02 21:45:42.725536 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:42.725409 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-h52vw" podStartSLOduration=1.024674946 podStartE2EDuration="2.725390924s" totalImagesPullingTime="1.700715978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:45:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:45:42.72401505 +0000 UTC m=+877.173853677" watchObservedRunningTime="2026-07-02 21:45:42.725390924 +0000 UTC m=+877.175229545" Jul 02 21:45:42.740613 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:42.740566 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" podStartSLOduration=1.516739971 podStartE2EDuration="3.740550787s" totalImagesPullingTime="2.223810816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:45:42.737895516 +0000 UTC m=+877.187734141" watchObservedRunningTime="2026-07-02 21:45:42.740550787 +0000 UTC m=+877.190389449" Jul 02 21:45:42.960957 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:42.960900 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" Jul 02 21:45:43.049552 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.049519 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtb4z\" (UniqueName: \"kubernetes.io/projected/ba75e276-953f-46d2-8ad8-510f8896b6c4-kube-api-access-vtb4z\") pod \"ba75e276-953f-46d2-8ad8-510f8896b6c4\" (UID: \"ba75e276-953f-46d2-8ad8-510f8896b6c4\") " Jul 02 21:45:43.051662 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.051636 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba75e276-953f-46d2-8ad8-510f8896b6c4-kube-api-access-vtb4z" (OuterVolumeSpecName: "kube-api-access-vtb4z") pod "ba75e276-953f-46d2-8ad8-510f8896b6c4" (UID: "ba75e276-953f-46d2-8ad8-510f8896b6c4"). InnerVolumeSpecName "kube-api-access-vtb4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:45:43.151244 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.151209 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtb4z\" (UniqueName: \"kubernetes.io/projected/ba75e276-953f-46d2-8ad8-510f8896b6c4-kube-api-access-vtb4z\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:43.717132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.717043 2547 generic.go:356] "Generic (PLEG): container finished" podID="ba75e276-953f-46d2-8ad8-510f8896b6c4" containerID="4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810" exitCode=0 Jul 02 21:45:43.717132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.717088 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" Jul 02 21:45:43.717630 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.717132 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" event={"ID":"ba75e276-953f-46d2-8ad8-510f8896b6c4","Type":"ContainerDied","Data":"4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810"} Jul 02 21:45:43.717630 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.717184 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dxrpc" event={"ID":"ba75e276-953f-46d2-8ad8-510f8896b6c4","Type":"ContainerDied","Data":"650e44b46bf58b92df0240b5efdb1557c361f5783e702b0f106395e59b38f3af"} Jul 02 21:45:43.717630 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.717212 2547 scope.go:117] "RemoveContainer" containerID="4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810" Jul 02 21:45:43.726860 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.726846 2547 scope.go:117] "RemoveContainer" containerID="4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810" Jul 02 21:45:43.727079 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:45:43.727062 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810\": container with ID starting with 4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810 not found: ID does not exist" containerID="4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810" Jul 02 21:45:43.727128 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.727087 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810"} err="failed to get container status \"4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810\": rpc error: code = NotFound desc = could not find container \"4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810\": container with ID starting with 4229d141b32d34e2100bb7dbf5d0fa02b6aba3f1f61f6e5965bd9485c0a3a810 not found: ID does not exist" Jul 02 21:45:43.738648 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.738625 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dxrpc"] Jul 02 21:45:43.742192 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:43.742172 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dxrpc"] Jul 02 21:45:44.008664 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:44.008580 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba75e276-953f-46d2-8ad8-510f8896b6c4" path="/var/lib/kubelet/pods/ba75e276-953f-46d2-8ad8-510f8896b6c4/volumes" Jul 02 21:45:50.512744 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:50.512708 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-h52vw" Jul 02 21:45:50.513171 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:50.512777 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-h52vw" Jul 02 21:45:50.535738 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:50.535711 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-h52vw" Jul 02 21:45:50.777590 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:50.777522 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-h52vw" Jul 02 21:45:55.031410 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.031367 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6"] Jul 02 21:45:55.031788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.031742 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba75e276-953f-46d2-8ad8-510f8896b6c4" containerName="registry-server" Jul 02 21:45:55.031788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.031752 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba75e276-953f-46d2-8ad8-510f8896b6c4" containerName="registry-server" Jul 02 21:45:55.031863 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.031822 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba75e276-953f-46d2-8ad8-510f8896b6c4" containerName="registry-server" Jul 02 21:45:55.036491 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.036475 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.038912 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.038893 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lsjvq\"" Jul 02 21:45:55.042626 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.042601 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6"] Jul 02 21:45:55.157227 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.157197 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-668dd\" (UniqueName: \"kubernetes.io/projected/3a03b391-2b07-42c0-8970-4ccc8abb58d3-kube-api-access-668dd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.157397 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.157236 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.157397 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.157332 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.257858 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.257821 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-668dd\" (UniqueName: \"kubernetes.io/projected/3a03b391-2b07-42c0-8970-4ccc8abb58d3-kube-api-access-668dd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.257858 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.257862 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.258076 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.257906 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.258278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.258261 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.258315 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.258291 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.266852 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.266823 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-668dd\" (UniqueName: \"kubernetes.io/projected/3a03b391-2b07-42c0-8970-4ccc8abb58d3-kube-api-access-668dd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.366155 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.366124 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:55.501180 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.501152 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6"] Jul 02 21:45:55.503424 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:45:55.503371 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a03b391_2b07_42c0_8970_4ccc8abb58d3.slice/crio-84e81e84f819b2b07f20485f66bbd350822ea8e1da5d020dceab85b06abc1e1f WatchSource:0}: Error finding container 84e81e84f819b2b07f20485f66bbd350822ea8e1da5d020dceab85b06abc1e1f: Status 404 returned error can't find the container with id 84e81e84f819b2b07f20485f66bbd350822ea8e1da5d020dceab85b06abc1e1f Jul 02 21:45:55.637425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.637340 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt"] Jul 02 21:45:55.640929 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.640908 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.648312 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.648287 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt"] Jul 02 21:45:55.762972 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.762939 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqxmp\" (UniqueName: \"kubernetes.io/projected/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-kube-api-access-tqxmp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.762972 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.762985 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.763219 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.763105 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.781940 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.781903 2547 generic.go:356] "Generic (PLEG): container finished" podID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerID="8faff79c9c660e38fa80f91b5fd121940cc7b893215bb963606081786be54581" exitCode=0 Jul 02 21:45:55.782074 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.781964 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" event={"ID":"3a03b391-2b07-42c0-8970-4ccc8abb58d3","Type":"ContainerDied","Data":"8faff79c9c660e38fa80f91b5fd121940cc7b893215bb963606081786be54581"} Jul 02 21:45:55.782074 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.782012 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" event={"ID":"3a03b391-2b07-42c0-8970-4ccc8abb58d3","Type":"ContainerStarted","Data":"84e81e84f819b2b07f20485f66bbd350822ea8e1da5d020dceab85b06abc1e1f"} Jul 02 21:45:55.863943 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.863911 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqxmp\" (UniqueName: \"kubernetes.io/projected/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-kube-api-access-tqxmp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.864095 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.863951 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.864095 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.863999 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.864318 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.864302 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.864365 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.864333 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.871525 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.871507 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqxmp\" (UniqueName: \"kubernetes.io/projected/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-kube-api-access-tqxmp\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:55.983496 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:55.983402 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:45:56.119314 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.119284 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt"] Jul 02 21:45:56.121137 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:45:56.121101 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c4706e3_9f16_4a3d_a8b9_85e133f940d8.slice/crio-d33ceb90fb14a4d12793ecee29acca670e90081e6ee2e1df55a93cc3fbcec7c5 WatchSource:0}: Error finding container d33ceb90fb14a4d12793ecee29acca670e90081e6ee2e1df55a93cc3fbcec7c5: Status 404 returned error can't find the container with id d33ceb90fb14a4d12793ecee29acca670e90081e6ee2e1df55a93cc3fbcec7c5 Jul 02 21:45:56.231299 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.231273 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj"] Jul 02 21:45:56.234901 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.234857 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.242247 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.242223 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj"] Jul 02 21:45:56.370679 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.370648 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7h9n\" (UniqueName: \"kubernetes.io/projected/d9063a72-ba5a-4bcf-9d99-2e87829a414c-kube-api-access-l7h9n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.370853 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.370685 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.370853 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.370729 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.472188 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.472158 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7h9n\" (UniqueName: \"kubernetes.io/projected/d9063a72-ba5a-4bcf-9d99-2e87829a414c-kube-api-access-l7h9n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.472188 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.472193 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.472365 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.472230 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.472607 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.472589 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.472650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.472620 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.481017 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.480975 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7h9n\" (UniqueName: \"kubernetes.io/projected/d9063a72-ba5a-4bcf-9d99-2e87829a414c-kube-api-access-l7h9n\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.577391 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.577360 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:45:56.631750 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.630636 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh"] Jul 02 21:45:56.635313 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.635287 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.644742 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.644715 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh"] Jul 02 21:45:56.719195 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.719172 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj"] Jul 02 21:45:56.721251 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:45:56.721229 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9063a72_ba5a_4bcf_9d99_2e87829a414c.slice/crio-e9282bdd15105b66ceb2f78b484ff418191ae014d212011d0bcd5ff49f1d3740 WatchSource:0}: Error finding container e9282bdd15105b66ceb2f78b484ff418191ae014d212011d0bcd5ff49f1d3740: Status 404 returned error can't find the container with id e9282bdd15105b66ceb2f78b484ff418191ae014d212011d0bcd5ff49f1d3740 Jul 02 21:45:56.774395 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.774359 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.774486 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.774459 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.774554 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.774489 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-227dl\" (UniqueName: \"kubernetes.io/projected/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-kube-api-access-227dl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.789594 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.789566 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" event={"ID":"d9063a72-ba5a-4bcf-9d99-2e87829a414c","Type":"ContainerStarted","Data":"67f1b58581cdc0af86e7f142b693f1bd6f13d22d10f899637d1e0f756cb449e7"} Jul 02 21:45:56.789711 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.789622 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" event={"ID":"d9063a72-ba5a-4bcf-9d99-2e87829a414c","Type":"ContainerStarted","Data":"e9282bdd15105b66ceb2f78b484ff418191ae014d212011d0bcd5ff49f1d3740"} Jul 02 21:45:56.791614 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.791592 2547 generic.go:356] "Generic (PLEG): container finished" podID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerID="5d31d1e80f04c8b6103c2446f4b84f1bbec690a0f02eb7ccaf85dcf84981041c" exitCode=0 Jul 02 21:45:56.791708 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.791673 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" event={"ID":"3a03b391-2b07-42c0-8970-4ccc8abb58d3","Type":"ContainerDied","Data":"5d31d1e80f04c8b6103c2446f4b84f1bbec690a0f02eb7ccaf85dcf84981041c"} Jul 02 21:45:56.802187 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.802154 2547 generic.go:356] "Generic (PLEG): container finished" podID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerID="d600555f85ec35ef035cc20eca6fe50ea2baaf21f4604eba9657a839bbe41df9" exitCode=0 Jul 02 21:45:56.802331 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.802259 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" event={"ID":"2c4706e3-9f16-4a3d-a8b9-85e133f940d8","Type":"ContainerDied","Data":"d600555f85ec35ef035cc20eca6fe50ea2baaf21f4604eba9657a839bbe41df9"} Jul 02 21:45:56.802726 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.802706 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" event={"ID":"2c4706e3-9f16-4a3d-a8b9-85e133f940d8","Type":"ContainerStarted","Data":"d33ceb90fb14a4d12793ecee29acca670e90081e6ee2e1df55a93cc3fbcec7c5"} Jul 02 21:45:56.875862 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.875829 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-227dl\" (UniqueName: \"kubernetes.io/projected/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-kube-api-access-227dl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.876008 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.875928 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.876008 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.875963 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.876363 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.876344 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.876450 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.876364 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.884291 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.884267 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-227dl\" (UniqueName: \"kubernetes.io/projected/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-kube-api-access-227dl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:56.952740 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:56.952701 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:45:57.089885 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.089860 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh"] Jul 02 21:45:57.091721 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:45:57.091694 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb2013a_c7e2_40b0_99c2_732eb2e19b5d.slice/crio-2318f8329986e3ef9969607e9f4d22053ef5ff33207ed553c68f3b4797ceae70 WatchSource:0}: Error finding container 2318f8329986e3ef9969607e9f4d22053ef5ff33207ed553c68f3b4797ceae70: Status 404 returned error can't find the container with id 2318f8329986e3ef9969607e9f4d22053ef5ff33207ed553c68f3b4797ceae70 Jul 02 21:45:57.809575 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.809527 2547 generic.go:356] "Generic (PLEG): container finished" podID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerID="67f1b58581cdc0af86e7f142b693f1bd6f13d22d10f899637d1e0f756cb449e7" exitCode=0 Jul 02 21:45:57.810055 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.809610 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" event={"ID":"d9063a72-ba5a-4bcf-9d99-2e87829a414c","Type":"ContainerDied","Data":"67f1b58581cdc0af86e7f142b693f1bd6f13d22d10f899637d1e0f756cb449e7"} Jul 02 21:45:57.812033 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.811927 2547 generic.go:356] "Generic (PLEG): container finished" podID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerID="65344cf9b0d4a06548dfe3087023acae6b33f274373ec3b6f3f70b52360f5677" exitCode=0 Jul 02 21:45:57.812033 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.811979 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" event={"ID":"3a03b391-2b07-42c0-8970-4ccc8abb58d3","Type":"ContainerDied","Data":"65344cf9b0d4a06548dfe3087023acae6b33f274373ec3b6f3f70b52360f5677"} Jul 02 21:45:57.813880 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.813860 2547 generic.go:356] "Generic (PLEG): container finished" podID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerID="cae6b4e9e434bfa3861a1eff0480a3704d0becdb0d020f562665a93f23ef474b" exitCode=0 Jul 02 21:45:57.813958 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.813891 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" event={"ID":"2c4706e3-9f16-4a3d-a8b9-85e133f940d8","Type":"ContainerDied","Data":"cae6b4e9e434bfa3861a1eff0480a3704d0becdb0d020f562665a93f23ef474b"} Jul 02 21:45:57.815450 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.815429 2547 generic.go:356] "Generic (PLEG): container finished" podID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerID="735c37946df3577e46767db653a90942b950c5b16b76aa111db97fc9b4fb752b" exitCode=0 Jul 02 21:45:57.815547 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.815518 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" event={"ID":"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d","Type":"ContainerDied","Data":"735c37946df3577e46767db653a90942b950c5b16b76aa111db97fc9b4fb752b"} Jul 02 21:45:57.815590 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:57.815573 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" event={"ID":"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d","Type":"ContainerStarted","Data":"2318f8329986e3ef9969607e9f4d22053ef5ff33207ed553c68f3b4797ceae70"} Jul 02 21:45:58.822453 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:58.822421 2547 generic.go:356] "Generic (PLEG): container finished" podID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerID="b2ff226787c4d79e19ce705bf54f9925286e90b6d743513560737d47cf118569" exitCode=0 Jul 02 21:45:58.822804 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:58.822494 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" event={"ID":"d9063a72-ba5a-4bcf-9d99-2e87829a414c","Type":"ContainerDied","Data":"b2ff226787c4d79e19ce705bf54f9925286e90b6d743513560737d47cf118569"} Jul 02 21:45:58.824828 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:58.824806 2547 generic.go:356] "Generic (PLEG): container finished" podID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerID="4d04f874f3fee226ba38d33e1fed5c1651850ecba6e5369b8a9ee8e6a20580bb" exitCode=0 Jul 02 21:45:58.824931 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:58.824870 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" event={"ID":"2c4706e3-9f16-4a3d-a8b9-85e133f940d8","Type":"ContainerDied","Data":"4d04f874f3fee226ba38d33e1fed5c1651850ecba6e5369b8a9ee8e6a20580bb"} Jul 02 21:45:58.826676 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:58.826654 2547 generic.go:356] "Generic (PLEG): container finished" podID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerID="be0e13579e506eb9a827e9efbdc8f6fb6133121af71309e1470e67a569560ce4" exitCode=0 Jul 02 21:45:58.826775 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:58.826736 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" event={"ID":"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d","Type":"ContainerDied","Data":"be0e13579e506eb9a827e9efbdc8f6fb6133121af71309e1470e67a569560ce4"} Jul 02 21:45:58.957918 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:58.957890 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:59.094601 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.094570 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-668dd\" (UniqueName: \"kubernetes.io/projected/3a03b391-2b07-42c0-8970-4ccc8abb58d3-kube-api-access-668dd\") pod \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " Jul 02 21:45:59.094750 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.094616 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-bundle\") pod \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " Jul 02 21:45:59.094790 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.094745 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-util\") pod \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\" (UID: \"3a03b391-2b07-42c0-8970-4ccc8abb58d3\") " Jul 02 21:45:59.095249 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.095222 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-bundle" (OuterVolumeSpecName: "bundle") pod "3a03b391-2b07-42c0-8970-4ccc8abb58d3" (UID: "3a03b391-2b07-42c0-8970-4ccc8abb58d3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:45:59.096667 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.096641 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a03b391-2b07-42c0-8970-4ccc8abb58d3-kube-api-access-668dd" (OuterVolumeSpecName: "kube-api-access-668dd") pod "3a03b391-2b07-42c0-8970-4ccc8abb58d3" (UID: "3a03b391-2b07-42c0-8970-4ccc8abb58d3"). InnerVolumeSpecName "kube-api-access-668dd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:45:59.100127 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.100097 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-util" (OuterVolumeSpecName: "util") pod "3a03b391-2b07-42c0-8970-4ccc8abb58d3" (UID: "3a03b391-2b07-42c0-8970-4ccc8abb58d3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:45:59.196067 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.196020 2547 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-util\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:59.196067 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.196062 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-668dd\" (UniqueName: \"kubernetes.io/projected/3a03b391-2b07-42c0-8970-4ccc8abb58d3-kube-api-access-668dd\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:59.196067 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.196073 2547 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a03b391-2b07-42c0-8970-4ccc8abb58d3-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:45:59.834505 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.834467 2547 generic.go:356] "Generic (PLEG): container finished" podID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerID="69b07273730eab7724eb77f728c48d3ba05ba33029aaedca5660ac901a7182ab" exitCode=0 Jul 02 21:45:59.834998 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.834552 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" event={"ID":"d9063a72-ba5a-4bcf-9d99-2e87829a414c","Type":"ContainerDied","Data":"69b07273730eab7724eb77f728c48d3ba05ba33029aaedca5660ac901a7182ab"} Jul 02 21:45:59.836279 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.836257 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" Jul 02 21:45:59.836279 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.836266 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6" event={"ID":"3a03b391-2b07-42c0-8970-4ccc8abb58d3","Type":"ContainerDied","Data":"84e81e84f819b2b07f20485f66bbd350822ea8e1da5d020dceab85b06abc1e1f"} Jul 02 21:45:59.836460 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.836307 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="84e81e84f819b2b07f20485f66bbd350822ea8e1da5d020dceab85b06abc1e1f" Jul 02 21:45:59.838321 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.838297 2547 generic.go:356] "Generic (PLEG): container finished" podID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerID="0e431b5640efa374ed1842fee3b2e12079d6084d8c6f8c77c34e76885e795d5e" exitCode=0 Jul 02 21:45:59.838573 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.838354 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" event={"ID":"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d","Type":"ContainerDied","Data":"0e431b5640efa374ed1842fee3b2e12079d6084d8c6f8c77c34e76885e795d5e"} Jul 02 21:45:59.969583 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:45:59.969559 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:46:00.003323 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.003292 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-util\") pod \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " Jul 02 21:46:00.003547 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.003348 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqxmp\" (UniqueName: \"kubernetes.io/projected/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-kube-api-access-tqxmp\") pod \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " Jul 02 21:46:00.003547 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.003419 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-bundle\") pod \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\" (UID: \"2c4706e3-9f16-4a3d-a8b9-85e133f940d8\") " Jul 02 21:46:00.003988 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.003959 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-bundle" (OuterVolumeSpecName: "bundle") pod "2c4706e3-9f16-4a3d-a8b9-85e133f940d8" (UID: "2c4706e3-9f16-4a3d-a8b9-85e133f940d8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:46:00.005889 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.005860 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-kube-api-access-tqxmp" (OuterVolumeSpecName: "kube-api-access-tqxmp") pod "2c4706e3-9f16-4a3d-a8b9-85e133f940d8" (UID: "2c4706e3-9f16-4a3d-a8b9-85e133f940d8"). InnerVolumeSpecName "kube-api-access-tqxmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:46:00.010243 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.010220 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-util" (OuterVolumeSpecName: "util") pod "2c4706e3-9f16-4a3d-a8b9-85e133f940d8" (UID: "2c4706e3-9f16-4a3d-a8b9-85e133f940d8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:46:00.105008 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.104914 2547 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:00.105008 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.104934 2547 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-util\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:00.105008 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.104944 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqxmp\" (UniqueName: \"kubernetes.io/projected/2c4706e3-9f16-4a3d-a8b9-85e133f940d8-kube-api-access-tqxmp\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:00.845834 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.845805 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" Jul 02 21:46:00.846232 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.845838 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt" event={"ID":"2c4706e3-9f16-4a3d-a8b9-85e133f940d8","Type":"ContainerDied","Data":"d33ceb90fb14a4d12793ecee29acca670e90081e6ee2e1df55a93cc3fbcec7c5"} Jul 02 21:46:00.846232 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.845881 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d33ceb90fb14a4d12793ecee29acca670e90081e6ee2e1df55a93cc3fbcec7c5" Jul 02 21:46:00.981045 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:00.981024 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:46:01.011445 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.011417 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:46:01.013459 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.013441 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-util\") pod \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " Jul 02 21:46:01.013538 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.013493 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7h9n\" (UniqueName: \"kubernetes.io/projected/d9063a72-ba5a-4bcf-9d99-2e87829a414c-kube-api-access-l7h9n\") pod \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " Jul 02 21:46:01.013590 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.013563 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-bundle\") pod \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\" (UID: \"d9063a72-ba5a-4bcf-9d99-2e87829a414c\") " Jul 02 21:46:01.014018 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.013989 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-bundle" (OuterVolumeSpecName: "bundle") pod "d9063a72-ba5a-4bcf-9d99-2e87829a414c" (UID: "d9063a72-ba5a-4bcf-9d99-2e87829a414c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:46:01.015770 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.015745 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9063a72-ba5a-4bcf-9d99-2e87829a414c-kube-api-access-l7h9n" (OuterVolumeSpecName: "kube-api-access-l7h9n") pod "d9063a72-ba5a-4bcf-9d99-2e87829a414c" (UID: "d9063a72-ba5a-4bcf-9d99-2e87829a414c"). InnerVolumeSpecName "kube-api-access-l7h9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:46:01.018979 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.018957 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-util" (OuterVolumeSpecName: "util") pod "d9063a72-ba5a-4bcf-9d99-2e87829a414c" (UID: "d9063a72-ba5a-4bcf-9d99-2e87829a414c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:46:01.114299 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.114203 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-util\") pod \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " Jul 02 21:46:01.114299 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.114250 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-227dl\" (UniqueName: \"kubernetes.io/projected/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-kube-api-access-227dl\") pod \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " Jul 02 21:46:01.114299 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.114297 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-bundle\") pod \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\" (UID: \"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d\") " Jul 02 21:46:01.114643 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.114485 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7h9n\" (UniqueName: \"kubernetes.io/projected/d9063a72-ba5a-4bcf-9d99-2e87829a414c-kube-api-access-l7h9n\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:01.114643 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.114496 2547 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:01.114643 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.114504 2547 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d9063a72-ba5a-4bcf-9d99-2e87829a414c-util\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:01.114951 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.114923 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-bundle" (OuterVolumeSpecName: "bundle") pod "ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" (UID: "ebb2013a-c7e2-40b0-99c2-732eb2e19b5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:46:01.116368 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.116346 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-kube-api-access-227dl" (OuterVolumeSpecName: "kube-api-access-227dl") pod "ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" (UID: "ebb2013a-c7e2-40b0-99c2-732eb2e19b5d"). InnerVolumeSpecName "kube-api-access-227dl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:46:01.119938 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.119913 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-util" (OuterVolumeSpecName: "util") pod "ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" (UID: "ebb2013a-c7e2-40b0-99c2-732eb2e19b5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:46:01.215365 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.215325 2547 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-util\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:01.215365 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.215358 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-227dl\" (UniqueName: \"kubernetes.io/projected/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-kube-api-access-227dl\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:01.215365 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.215372 2547 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebb2013a-c7e2-40b0-99c2-732eb2e19b5d-bundle\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:01.852516 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.852483 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" Jul 02 21:46:01.852907 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.852512 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj" event={"ID":"d9063a72-ba5a-4bcf-9d99-2e87829a414c","Type":"ContainerDied","Data":"e9282bdd15105b66ceb2f78b484ff418191ae014d212011d0bcd5ff49f1d3740"} Jul 02 21:46:01.852907 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.852561 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9282bdd15105b66ceb2f78b484ff418191ae014d212011d0bcd5ff49f1d3740" Jul 02 21:46:01.854265 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.854242 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" event={"ID":"ebb2013a-c7e2-40b0-99c2-732eb2e19b5d","Type":"ContainerDied","Data":"2318f8329986e3ef9969607e9f4d22053ef5ff33207ed553c68f3b4797ceae70"} Jul 02 21:46:01.854420 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.854273 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2318f8329986e3ef9969607e9f4d22053ef5ff33207ed553c68f3b4797ceae70" Jul 02 21:46:01.854420 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:01.854277 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh" Jul 02 21:46:05.933687 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:05.933661 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:46:05.934134 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:05.934057 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:46:05.940996 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:05.940967 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:46:05.941126 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:05.941012 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:46:11.189015 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.188976 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h"] Jul 02 21:46:11.189424 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189409 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerName="pull" Jul 02 21:46:11.189472 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189427 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerName="pull" Jul 02 21:46:11.189472 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189440 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerName="extract" Jul 02 21:46:11.189472 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189447 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerName="extract" Jul 02 21:46:11.189472 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189459 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerName="pull" Jul 02 21:46:11.189472 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189464 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerName="pull" Jul 02 21:46:11.189472 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189471 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerName="pull" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189476 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerName="pull" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189488 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerName="util" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189493 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerName="util" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189497 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerName="pull" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189502 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerName="pull" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189512 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerName="extract" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189520 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerName="extract" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189526 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerName="util" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189531 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerName="util" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189536 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerName="util" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189541 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerName="util" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189551 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerName="util" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189556 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerName="util" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189562 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerName="extract" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189566 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerName="extract" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189572 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerName="extract" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189578 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerName="extract" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189638 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a03b391-2b07-42c0-8970-4ccc8abb58d3" containerName="extract" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189647 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebb2013a-c7e2-40b0-99c2-732eb2e19b5d" containerName="extract" Jul 02 21:46:11.189653 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189653 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c4706e3-9f16-4a3d-a8b9-85e133f940d8" containerName="extract" Jul 02 21:46:11.190217 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.189663 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9063a72-ba5a-4bcf-9d99-2e87829a414c" containerName="extract" Jul 02 21:46:11.194813 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.194798 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:11.197325 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.197301 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m4chd\"" Jul 02 21:46:11.204234 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.204212 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h"] Jul 02 21:46:11.304328 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.304294 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwxng\" (UniqueName: \"kubernetes.io/projected/86f9e7fe-7b5f-4d4f-9979-89264d36170b-kube-api-access-vwxng\") pod \"limitador-operator-controller-manager-859d7b78bd-bpl5h\" (UID: \"86f9e7fe-7b5f-4d4f-9979-89264d36170b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:11.405773 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.405732 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwxng\" (UniqueName: \"kubernetes.io/projected/86f9e7fe-7b5f-4d4f-9979-89264d36170b-kube-api-access-vwxng\") pod \"limitador-operator-controller-manager-859d7b78bd-bpl5h\" (UID: \"86f9e7fe-7b5f-4d4f-9979-89264d36170b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:11.414476 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.414443 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwxng\" (UniqueName: \"kubernetes.io/projected/86f9e7fe-7b5f-4d4f-9979-89264d36170b-kube-api-access-vwxng\") pod \"limitador-operator-controller-manager-859d7b78bd-bpl5h\" (UID: \"86f9e7fe-7b5f-4d4f-9979-89264d36170b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:11.506125 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.506036 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:11.646252 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.646227 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h"] Jul 02 21:46:11.647688 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:46:11.647655 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86f9e7fe_7b5f_4d4f_9979_89264d36170b.slice/crio-2d8b466eec733b872e77559e767490b67a14417ff82672c79e9f6bf771ecb68c WatchSource:0}: Error finding container 2d8b466eec733b872e77559e767490b67a14417ff82672c79e9f6bf771ecb68c: Status 404 returned error can't find the container with id 2d8b466eec733b872e77559e767490b67a14417ff82672c79e9f6bf771ecb68c Jul 02 21:46:11.906117 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:11.906084 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" event={"ID":"86f9e7fe-7b5f-4d4f-9979-89264d36170b","Type":"ContainerStarted","Data":"2d8b466eec733b872e77559e767490b67a14417ff82672c79e9f6bf771ecb68c"} Jul 02 21:46:13.921999 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:13.921963 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" event={"ID":"86f9e7fe-7b5f-4d4f-9979-89264d36170b","Type":"ContainerStarted","Data":"dea768b987a63256753e2d73de52a3ab1f6128fca2e8dc2f1d31bf06fd2a6ae2"} Jul 02 21:46:13.922424 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:13.922102 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:13.949001 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:13.948950 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" podStartSLOduration=1.496103997 podStartE2EDuration="2.94893621s" totalImagesPullingTime="1.452832213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:13.946851549 +0000 UTC m=+908.396690175" watchObservedRunningTime="2026-07-02 21:46:13.94893621 +0000 UTC m=+908.398774837" Jul 02 21:46:16.269854 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.269824 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl"] Jul 02 21:46:16.273286 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.273266 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:16.276147 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.276127 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nkt56\"" Jul 02 21:46:16.287582 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.287559 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl"] Jul 02 21:46:16.349242 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.349214 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msw7k\" (UniqueName: \"kubernetes.io/projected/76418d12-67a0-4785-9c86-ef80a37a10be-kube-api-access-msw7k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" (UID: \"76418d12-67a0-4785-9c86-ef80a37a10be\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:16.349425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.349305 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76418d12-67a0-4785-9c86-ef80a37a10be-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" (UID: \"76418d12-67a0-4785-9c86-ef80a37a10be\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:16.450572 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.450534 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76418d12-67a0-4785-9c86-ef80a37a10be-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" (UID: \"76418d12-67a0-4785-9c86-ef80a37a10be\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:16.450741 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.450584 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msw7k\" (UniqueName: \"kubernetes.io/projected/76418d12-67a0-4785-9c86-ef80a37a10be-kube-api-access-msw7k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" (UID: \"76418d12-67a0-4785-9c86-ef80a37a10be\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:16.450917 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.450895 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76418d12-67a0-4785-9c86-ef80a37a10be-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" (UID: \"76418d12-67a0-4785-9c86-ef80a37a10be\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:16.465029 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.465006 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msw7k\" (UniqueName: \"kubernetes.io/projected/76418d12-67a0-4785-9c86-ef80a37a10be-kube-api-access-msw7k\") pod \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" (UID: \"76418d12-67a0-4785-9c86-ef80a37a10be\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:16.583727 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.583693 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:16.724807 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.724497 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl"] Jul 02 21:46:16.727786 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:46:16.727756 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76418d12_67a0_4785_9c86_ef80a37a10be.slice/crio-58bc103e97e9d79d0dbb2ee2db2f434db9048d05b43d0c1310b31bf38bf6a856 WatchSource:0}: Error finding container 58bc103e97e9d79d0dbb2ee2db2f434db9048d05b43d0c1310b31bf38bf6a856: Status 404 returned error can't find the container with id 58bc103e97e9d79d0dbb2ee2db2f434db9048d05b43d0c1310b31bf38bf6a856 Jul 02 21:46:16.940161 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:16.940082 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" event={"ID":"76418d12-67a0-4785-9c86-ef80a37a10be","Type":"ContainerStarted","Data":"58bc103e97e9d79d0dbb2ee2db2f434db9048d05b43d0c1310b31bf38bf6a856"} Jul 02 21:46:18.987493 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:18.987457 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6"] Jul 02 21:46:18.991525 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:18.991504 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" Jul 02 21:46:18.994340 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:18.994322 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-qdpn6\"" Jul 02 21:46:19.001573 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:19.001552 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6"] Jul 02 21:46:19.075903 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:19.075862 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlj2p\" (UniqueName: \"kubernetes.io/projected/2295ca83-9de0-4868-9c49-05324f1abee2-kube-api-access-jlj2p\") pod \"authorino-operator-75cc8d5cb7-mb7r6\" (UID: \"2295ca83-9de0-4868-9c49-05324f1abee2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" Jul 02 21:46:19.177918 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:19.177876 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlj2p\" (UniqueName: \"kubernetes.io/projected/2295ca83-9de0-4868-9c49-05324f1abee2-kube-api-access-jlj2p\") pod \"authorino-operator-75cc8d5cb7-mb7r6\" (UID: \"2295ca83-9de0-4868-9c49-05324f1abee2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" Jul 02 21:46:19.191686 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:19.191648 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlj2p\" (UniqueName: \"kubernetes.io/projected/2295ca83-9de0-4868-9c49-05324f1abee2-kube-api-access-jlj2p\") pod \"authorino-operator-75cc8d5cb7-mb7r6\" (UID: \"2295ca83-9de0-4868-9c49-05324f1abee2\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" Jul 02 21:46:19.304065 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:19.303916 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" Jul 02 21:46:19.471723 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:19.469811 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6"] Jul 02 21:46:19.959640 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:19.959598 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" event={"ID":"2295ca83-9de0-4868-9c49-05324f1abee2","Type":"ContainerStarted","Data":"d33547b545a3b4cdc09e8d127cd17ea7dea4c07f1d1e5fbfa5838f1a60d9369c"} Jul 02 21:46:22.402288 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:22.402251 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v"] Jul 02 21:46:22.406442 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:22.406421 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" Jul 02 21:46:22.409549 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:22.409302 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jztjm\"" Jul 02 21:46:22.409549 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:22.409354 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 02 21:46:22.410970 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:22.410902 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v"] Jul 02 21:46:22.508416 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:22.508353 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkrqn\" (UniqueName: \"kubernetes.io/projected/1948f8b4-17d8-473d-be76-ee7dd353e798-kube-api-access-hkrqn\") pod \"dns-operator-controller-manager-864dd78c4c-bfp6v\" (UID: \"1948f8b4-17d8-473d-be76-ee7dd353e798\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" Jul 02 21:46:22.609696 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:22.609661 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkrqn\" (UniqueName: \"kubernetes.io/projected/1948f8b4-17d8-473d-be76-ee7dd353e798-kube-api-access-hkrqn\") pod \"dns-operator-controller-manager-864dd78c4c-bfp6v\" (UID: \"1948f8b4-17d8-473d-be76-ee7dd353e798\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" Jul 02 21:46:22.624543 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:22.624518 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkrqn\" (UniqueName: \"kubernetes.io/projected/1948f8b4-17d8-473d-be76-ee7dd353e798-kube-api-access-hkrqn\") pod \"dns-operator-controller-manager-864dd78c4c-bfp6v\" (UID: \"1948f8b4-17d8-473d-be76-ee7dd353e798\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" Jul 02 21:46:22.720331 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:22.720236 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" Jul 02 21:46:23.114560 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:23.114529 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v"] Jul 02 21:46:23.116270 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:46:23.116247 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1948f8b4_17d8_473d_be76_ee7dd353e798.slice/crio-3ffaefc52608367237b8cf13a3f89250df8ed6e929a52240e181656e53d622f8 WatchSource:0}: Error finding container 3ffaefc52608367237b8cf13a3f89250df8ed6e929a52240e181656e53d622f8: Status 404 returned error can't find the container with id 3ffaefc52608367237b8cf13a3f89250df8ed6e929a52240e181656e53d622f8 Jul 02 21:46:23.992411 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:23.992281 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" event={"ID":"76418d12-67a0-4785-9c86-ef80a37a10be","Type":"ContainerStarted","Data":"a8ab212b26bfa938489390632ee1f170cad266c1a4f1f23d017c6c87b35b1034"} Jul 02 21:46:23.992411 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:23.992351 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:23.993962 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:23.993936 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" event={"ID":"1948f8b4-17d8-473d-be76-ee7dd353e798","Type":"ContainerStarted","Data":"3ffaefc52608367237b8cf13a3f89250df8ed6e929a52240e181656e53d622f8"} Jul 02 21:46:23.995911 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:23.995886 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" event={"ID":"2295ca83-9de0-4868-9c49-05324f1abee2","Type":"ContainerStarted","Data":"7131edaa000b5b8cb2303ff3416168277ba20785289dc1404cf3c606ef9f28d5"} Jul 02 21:46:23.996029 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:23.995979 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" Jul 02 21:46:24.014499 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:24.014439 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" podStartSLOduration=1.7585842440000001 podStartE2EDuration="8.014417982s" totalImagesPullingTime="6.255833738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:24.011957355 +0000 UTC m=+918.461795980" watchObservedRunningTime="2026-07-02 21:46:24.014417982 +0000 UTC m=+918.464256610" Jul 02 21:46:24.031733 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:24.031668 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" podStartSLOduration=2.535195502 podStartE2EDuration="6.03164697s" totalImagesPullingTime="3.496451468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:24.030655729 +0000 UTC m=+918.480494352" watchObservedRunningTime="2026-07-02 21:46:24.03164697 +0000 UTC m=+918.481485594" Jul 02 21:46:24.929951 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:24.929922 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:25.004007 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:25.003965 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" event={"ID":"1948f8b4-17d8-473d-be76-ee7dd353e798","Type":"ContainerStarted","Data":"7cf1c0ae74c24073971a27e3d023fd9a460ac390fbe5da9e7876d873e63efc1f"} Jul 02 21:46:25.004412 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:25.004025 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" Jul 02 21:46:25.023005 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:25.022954 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" podStartSLOduration=1.30599032 podStartE2EDuration="3.022938232s" totalImagesPullingTime="1.716947912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:25.022145652 +0000 UTC m=+919.471984279" watchObservedRunningTime="2026-07-02 21:46:25.022938232 +0000 UTC m=+919.472776852" Jul 02 21:46:26.025423 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.025374 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6"] Jul 02 21:46:26.029177 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.029159 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.031742 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.031708 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 02 21:46:26.031848 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.031796 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lsjvq\"" Jul 02 21:46:26.031928 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.031914 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 02 21:46:26.037117 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.037088 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6"] Jul 02 21:46:26.142627 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.142596 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ea8b1c9-5aea-44c5-a101-054d8ee19668-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.142627 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.142630 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46thn\" (UniqueName: \"kubernetes.io/projected/8ea8b1c9-5aea-44c5-a101-054d8ee19668-kube-api-access-46thn\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.142835 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.142721 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea8b1c9-5aea-44c5-a101-054d8ee19668-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.243767 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.243729 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ea8b1c9-5aea-44c5-a101-054d8ee19668-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.243944 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.243774 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46thn\" (UniqueName: \"kubernetes.io/projected/8ea8b1c9-5aea-44c5-a101-054d8ee19668-kube-api-access-46thn\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.243944 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.243822 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea8b1c9-5aea-44c5-a101-054d8ee19668-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.243944 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:26.243936 2547 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 02 21:46:26.244112 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:26.244003 2547 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea8b1c9-5aea-44c5-a101-054d8ee19668-plugin-serving-cert podName:8ea8b1c9-5aea-44c5-a101-054d8ee19668 nodeName:}" failed. No retries permitted until 2026-07-02 21:46:26.743981753 +0000 UTC m=+921.193820360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/8ea8b1c9-5aea-44c5-a101-054d8ee19668-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-9ngp6" (UID: "8ea8b1c9-5aea-44c5-a101-054d8ee19668") : secret "plugin-serving-cert" not found Jul 02 21:46:26.244416 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.244365 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8ea8b1c9-5aea-44c5-a101-054d8ee19668-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.254481 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.254460 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46thn\" (UniqueName: \"kubernetes.io/projected/8ea8b1c9-5aea-44c5-a101-054d8ee19668-kube-api-access-46thn\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.748039 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.748002 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea8b1c9-5aea-44c5-a101-054d8ee19668-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.750504 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.750480 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ea8b1c9-5aea-44c5-a101-054d8ee19668-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9ngp6\" (UID: \"8ea8b1c9-5aea-44c5-a101-054d8ee19668\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:26.939607 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:26.939570 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" Jul 02 21:46:27.079615 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:27.079588 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6"] Jul 02 21:46:27.081006 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:46:27.080977 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ea8b1c9_5aea_44c5_a101_054d8ee19668.slice/crio-18e2bf852063f1137e2067ee73a47f9f54cf8e69a1b28050d491291746ca5003 WatchSource:0}: Error finding container 18e2bf852063f1137e2067ee73a47f9f54cf8e69a1b28050d491291746ca5003: Status 404 returned error can't find the container with id 18e2bf852063f1137e2067ee73a47f9f54cf8e69a1b28050d491291746ca5003 Jul 02 21:46:28.023646 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:28.023609 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" event={"ID":"8ea8b1c9-5aea-44c5-a101-054d8ee19668","Type":"ContainerStarted","Data":"18e2bf852063f1137e2067ee73a47f9f54cf8e69a1b28050d491291746ca5003"} Jul 02 21:46:35.006606 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:35.006574 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mb7r6" Jul 02 21:46:35.007158 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:35.006666 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:36.013187 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.013156 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-bfp6v" Jul 02 21:46:36.782789 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.782752 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl"] Jul 02 21:46:36.783003 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.782972 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" containerName="manager" containerID="cri-o://a8ab212b26bfa938489390632ee1f170cad266c1a4f1f23d017c6c87b35b1034" gracePeriod=2 Jul 02 21:46:36.799028 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.798999 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl"] Jul 02 21:46:36.819681 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.819653 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h"] Jul 02 21:46:36.819893 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.819869 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" podUID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" containerName="manager" containerID="cri-o://dea768b987a63256753e2d73de52a3ab1f6128fca2e8dc2f1d31bf06fd2a6ae2" gracePeriod=2 Jul 02 21:46:36.824101 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:36.823798 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:36.828739 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.828713 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll"] Jul 02 21:46:36.829323 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.829302 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" containerName="manager" Jul 02 21:46:36.829323 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.829323 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" containerName="manager" Jul 02 21:46:36.829498 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.829470 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" containerName="manager" Jul 02 21:46:36.841228 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.841191 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h"] Jul 02 21:46:36.841347 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.841302 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:36.841762 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:36.841534 2547 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"86f9e7fe-7b5f-4d4f-9979-89264d36170b\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-bpl5h\": pods \"limitador-operator-controller-manager-859d7b78bd-bpl5h\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:36.844003 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:36.843975 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bpl5h\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:36.846216 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:36.846189 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:36.859192 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.859163 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll"] Jul 02 21:46:36.862980 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.862592 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt"] Jul 02 21:46:36.863072 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.863059 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" containerName="manager" Jul 02 21:46:36.863116 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.863073 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" containerName="manager" Jul 02 21:46:36.863150 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.863141 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" containerName="manager" Jul 02 21:46:36.866663 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.866637 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" Jul 02 21:46:36.889255 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.889224 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt"] Jul 02 21:46:36.889514 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:36.889483 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bpl5h\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:36.932664 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:36.932610 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:36.946049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.946008 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfhjf\" (UniqueName: \"kubernetes.io/projected/51eaab2f-160d-404d-bf0c-aafccbb0654e-kube-api-access-tfhjf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pzfll\" (UID: \"51eaab2f-160d-404d-bf0c-aafccbb0654e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:36.946209 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:36.946163 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51eaab2f-160d-404d-bf0c-aafccbb0654e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pzfll\" (UID: \"51eaab2f-160d-404d-bf0c-aafccbb0654e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:37.046905 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:37.046871 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51eaab2f-160d-404d-bf0c-aafccbb0654e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pzfll\" (UID: \"51eaab2f-160d-404d-bf0c-aafccbb0654e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:37.047339 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:37.046951 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfhjf\" (UniqueName: \"kubernetes.io/projected/51eaab2f-160d-404d-bf0c-aafccbb0654e-kube-api-access-tfhjf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pzfll\" (UID: \"51eaab2f-160d-404d-bf0c-aafccbb0654e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:37.047339 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:37.047004 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf2sd\" (UniqueName: \"kubernetes.io/projected/6ae1d647-c86f-4aaa-a478-f0dd527144c6-kube-api-access-mf2sd\") pod \"limitador-operator-controller-manager-859d7b78bd-djzzt\" (UID: \"6ae1d647-c86f-4aaa-a478-f0dd527144c6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" Jul 02 21:46:37.047339 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:37.047328 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51eaab2f-160d-404d-bf0c-aafccbb0654e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pzfll\" (UID: \"51eaab2f-160d-404d-bf0c-aafccbb0654e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:37.057600 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:37.057569 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfhjf\" (UniqueName: \"kubernetes.io/projected/51eaab2f-160d-404d-bf0c-aafccbb0654e-kube-api-access-tfhjf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-pzfll\" (UID: \"51eaab2f-160d-404d-bf0c-aafccbb0654e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:37.147891 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:37.147765 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf2sd\" (UniqueName: \"kubernetes.io/projected/6ae1d647-c86f-4aaa-a478-f0dd527144c6-kube-api-access-mf2sd\") pod \"limitador-operator-controller-manager-859d7b78bd-djzzt\" (UID: \"6ae1d647-c86f-4aaa-a478-f0dd527144c6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" Jul 02 21:46:37.162239 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:37.162206 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf2sd\" (UniqueName: \"kubernetes.io/projected/6ae1d647-c86f-4aaa-a478-f0dd527144c6-kube-api-access-mf2sd\") pod \"limitador-operator-controller-manager-859d7b78bd-djzzt\" (UID: \"6ae1d647-c86f-4aaa-a478-f0dd527144c6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" Jul 02 21:46:37.204848 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:37.204817 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:37.216348 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:37.216321 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" Jul 02 21:46:49.166221 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.166007 2547 generic.go:356] "Generic (PLEG): container finished" podID="76418d12-67a0-4785-9c86-ef80a37a10be" containerID="a8ab212b26bfa938489390632ee1f170cad266c1a4f1f23d017c6c87b35b1034" exitCode=0 Jul 02 21:46:49.169641 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.169609 2547 generic.go:356] "Generic (PLEG): container finished" podID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" containerID="dea768b987a63256753e2d73de52a3ab1f6128fca2e8dc2f1d31bf06fd2a6ae2" exitCode=0 Jul 02 21:46:49.232788 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.232765 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll"] Jul 02 21:46:49.234434 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:46:49.234371 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51eaab2f_160d_404d_bf0c_aafccbb0654e.slice/crio-32bbeefe715d1f3deefc4e4783fd17789e7f0476e7ab1fa1148472c5d8aa4448 WatchSource:0}: Error finding container 32bbeefe715d1f3deefc4e4783fd17789e7f0476e7ab1fa1148472c5d8aa4448: Status 404 returned error can't find the container with id 32bbeefe715d1f3deefc4e4783fd17789e7f0476e7ab1fa1148472c5d8aa4448 Jul 02 21:46:49.238625 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.238543 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:49.240937 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:49.240907 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:49.241774 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.241752 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:49.243858 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:49.243832 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:49.245931 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:49.245909 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bpl5h\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:49.258138 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.258114 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt"] Jul 02 21:46:49.259056 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:46:49.259033 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae1d647_c86f_4aaa_a478_f0dd527144c6.slice/crio-fa5a989af80a1d603e19040122f744745064be36f196ee30afabc8e1066752a0 WatchSource:0}: Error finding container fa5a989af80a1d603e19040122f744745064be36f196ee30afabc8e1066752a0: Status 404 returned error can't find the container with id fa5a989af80a1d603e19040122f744745064be36f196ee30afabc8e1066752a0 Jul 02 21:46:49.266332 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.266305 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76418d12-67a0-4785-9c86-ef80a37a10be-extensions-socket-volume\") pod \"76418d12-67a0-4785-9c86-ef80a37a10be\" (UID: \"76418d12-67a0-4785-9c86-ef80a37a10be\") " Jul 02 21:46:49.266459 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.266353 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msw7k\" (UniqueName: \"kubernetes.io/projected/76418d12-67a0-4785-9c86-ef80a37a10be-kube-api-access-msw7k\") pod \"76418d12-67a0-4785-9c86-ef80a37a10be\" (UID: \"76418d12-67a0-4785-9c86-ef80a37a10be\") " Jul 02 21:46:49.266542 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.266525 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwxng\" (UniqueName: \"kubernetes.io/projected/86f9e7fe-7b5f-4d4f-9979-89264d36170b-kube-api-access-vwxng\") pod \"86f9e7fe-7b5f-4d4f-9979-89264d36170b\" (UID: \"86f9e7fe-7b5f-4d4f-9979-89264d36170b\") " Jul 02 21:46:49.266875 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.266855 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76418d12-67a0-4785-9c86-ef80a37a10be-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "76418d12-67a0-4785-9c86-ef80a37a10be" (UID: "76418d12-67a0-4785-9c86-ef80a37a10be"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:46:49.268301 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.268275 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76418d12-67a0-4785-9c86-ef80a37a10be-kube-api-access-msw7k" (OuterVolumeSpecName: "kube-api-access-msw7k") pod "76418d12-67a0-4785-9c86-ef80a37a10be" (UID: "76418d12-67a0-4785-9c86-ef80a37a10be"). InnerVolumeSpecName "kube-api-access-msw7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:46:49.268405 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.268365 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86f9e7fe-7b5f-4d4f-9979-89264d36170b-kube-api-access-vwxng" (OuterVolumeSpecName: "kube-api-access-vwxng") pod "86f9e7fe-7b5f-4d4f-9979-89264d36170b" (UID: "86f9e7fe-7b5f-4d4f-9979-89264d36170b"). InnerVolumeSpecName "kube-api-access-vwxng". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:46:49.367879 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.367846 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwxng\" (UniqueName: \"kubernetes.io/projected/86f9e7fe-7b5f-4d4f-9979-89264d36170b-kube-api-access-vwxng\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:49.367879 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.367873 2547 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76418d12-67a0-4785-9c86-ef80a37a10be-extensions-socket-volume\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:49.368089 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:49.367887 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msw7k\" (UniqueName: \"kubernetes.io/projected/76418d12-67a0-4785-9c86-ef80a37a10be-kube-api-access-msw7k\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:50.009650 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.009615 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" path="/var/lib/kubelet/pods/76418d12-67a0-4785-9c86-ef80a37a10be/volumes" Jul 02 21:46:50.010009 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.009995 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" path="/var/lib/kubelet/pods/86f9e7fe-7b5f-4d4f-9979-89264d36170b/volumes" Jul 02 21:46:50.177079 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.177050 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:50.177570 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.177079 2547 scope.go:117] "RemoveContainer" containerID="a8ab212b26bfa938489390632ee1f170cad266c1a4f1f23d017c6c87b35b1034" Jul 02 21:46:50.178983 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.178953 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" event={"ID":"8ea8b1c9-5aea-44c5-a101-054d8ee19668","Type":"ContainerStarted","Data":"15212a6e3bb8cc755653056b19e70da3412d310f9de92fed5a4aedda94130899"} Jul 02 21:46:50.179675 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:50.179650 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:50.180973 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.180949 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" event={"ID":"6ae1d647-c86f-4aaa-a478-f0dd527144c6","Type":"ContainerStarted","Data":"4a62d88934a5324f4a8b4c38caffebf3e9ebac3f1ca555bb13cfd76c8c505426"} Jul 02 21:46:50.181096 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.181008 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" event={"ID":"6ae1d647-c86f-4aaa-a478-f0dd527144c6","Type":"ContainerStarted","Data":"fa5a989af80a1d603e19040122f744745064be36f196ee30afabc8e1066752a0"} Jul 02 21:46:50.181096 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.181039 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" Jul 02 21:46:50.181874 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:50.181844 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:50.182665 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.182641 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" event={"ID":"51eaab2f-160d-404d-bf0c-aafccbb0654e","Type":"ContainerStarted","Data":"6c912cf5da5b47aa8b4617ae42c745df05b07e60cb787fbb3f1242ed969ba6d8"} Jul 02 21:46:50.182764 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.182687 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" event={"ID":"51eaab2f-160d-404d-bf0c-aafccbb0654e","Type":"ContainerStarted","Data":"32bbeefe715d1f3deefc4e4783fd17789e7f0476e7ab1fa1148472c5d8aa4448"} Jul 02 21:46:50.182826 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.182769 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:50.184003 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.183984 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:50.187304 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.187285 2547 scope.go:117] "RemoveContainer" containerID="dea768b987a63256753e2d73de52a3ab1f6128fca2e8dc2f1d31bf06fd2a6ae2" Jul 02 21:46:50.203359 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.203299 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9ngp6" podStartSLOduration=2.136514139 podStartE2EDuration="24.203283432s" totalImagesPullingTime="22.066769293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:50.201463421 +0000 UTC m=+944.651302070" watchObservedRunningTime="2026-07-02 21:46:50.203283432 +0000 UTC m=+944.653122056" Jul 02 21:46:50.203667 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:50.203638 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-bpl5h\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="86f9e7fe-7b5f-4d4f-9979-89264d36170b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-bpl5h" Jul 02 21:46:50.227580 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:46:50.227540 2547 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-98ggl\" is forbidden: User \"system:node:ip-10-0-137-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-35.ec2.internal' and this object" podUID="76418d12-67a0-4785-9c86-ef80a37a10be" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-98ggl" Jul 02 21:46:50.228204 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.228153 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" podStartSLOduration=14.228134595 podStartE2EDuration="14.228134595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:50.224072736 +0000 UTC m=+944.673911385" watchObservedRunningTime="2026-07-02 21:46:50.228134595 +0000 UTC m=+944.677973227" Jul 02 21:46:50.246810 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:50.246756 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" podStartSLOduration=14.246739498 podStartE2EDuration="14.246739498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:46:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:46:50.244729508 +0000 UTC m=+944.694568133" watchObservedRunningTime="2026-07-02 21:46:50.246739498 +0000 UTC m=+944.696578179" Jul 02 21:46:54.010893 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.010849 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll"] Jul 02 21:46:54.011397 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.011041 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" podUID="51eaab2f-160d-404d-bf0c-aafccbb0654e" containerName="manager" containerID="cri-o://6c912cf5da5b47aa8b4617ae42c745df05b07e60cb787fbb3f1242ed969ba6d8" gracePeriod=10 Jul 02 21:46:54.013143 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.013119 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:54.211673 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.211512 2547 generic.go:356] "Generic (PLEG): container finished" podID="51eaab2f-160d-404d-bf0c-aafccbb0654e" containerID="6c912cf5da5b47aa8b4617ae42c745df05b07e60cb787fbb3f1242ed969ba6d8" exitCode=0 Jul 02 21:46:54.211807 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.211720 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" event={"ID":"51eaab2f-160d-404d-bf0c-aafccbb0654e","Type":"ContainerDied","Data":"6c912cf5da5b47aa8b4617ae42c745df05b07e60cb787fbb3f1242ed969ba6d8"} Jul 02 21:46:54.313437 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.313411 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:54.417195 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.417161 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tfhjf\" (UniqueName: \"kubernetes.io/projected/51eaab2f-160d-404d-bf0c-aafccbb0654e-kube-api-access-tfhjf\") pod \"51eaab2f-160d-404d-bf0c-aafccbb0654e\" (UID: \"51eaab2f-160d-404d-bf0c-aafccbb0654e\") " Jul 02 21:46:54.417195 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.417199 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51eaab2f-160d-404d-bf0c-aafccbb0654e-extensions-socket-volume\") pod \"51eaab2f-160d-404d-bf0c-aafccbb0654e\" (UID: \"51eaab2f-160d-404d-bf0c-aafccbb0654e\") " Jul 02 21:46:54.417631 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.417604 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51eaab2f-160d-404d-bf0c-aafccbb0654e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "51eaab2f-160d-404d-bf0c-aafccbb0654e" (UID: "51eaab2f-160d-404d-bf0c-aafccbb0654e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 21:46:54.419132 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.419104 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51eaab2f-160d-404d-bf0c-aafccbb0654e-kube-api-access-tfhjf" (OuterVolumeSpecName: "kube-api-access-tfhjf") pod "51eaab2f-160d-404d-bf0c-aafccbb0654e" (UID: "51eaab2f-160d-404d-bf0c-aafccbb0654e"). InnerVolumeSpecName "kube-api-access-tfhjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:46:54.518083 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.518052 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tfhjf\" (UniqueName: \"kubernetes.io/projected/51eaab2f-160d-404d-bf0c-aafccbb0654e-kube-api-access-tfhjf\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:54.518083 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:54.518080 2547 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/51eaab2f-160d-404d-bf0c-aafccbb0654e-extensions-socket-volume\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:46:55.220340 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:55.220314 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" Jul 02 21:46:55.220720 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:55.220314 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll" event={"ID":"51eaab2f-160d-404d-bf0c-aafccbb0654e","Type":"ContainerDied","Data":"32bbeefe715d1f3deefc4e4783fd17789e7f0476e7ab1fa1148472c5d8aa4448"} Jul 02 21:46:55.220720 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:55.220448 2547 scope.go:117] "RemoveContainer" containerID="6c912cf5da5b47aa8b4617ae42c745df05b07e60cb787fbb3f1242ed969ba6d8" Jul 02 21:46:55.246102 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:55.246063 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll"] Jul 02 21:46:55.249973 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:55.249944 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-pzfll"] Jul 02 21:46:56.010184 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:46:56.010151 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51eaab2f-160d-404d-bf0c-aafccbb0654e" path="/var/lib/kubelet/pods/51eaab2f-160d-404d-bf0c-aafccbb0654e/volumes" Jul 02 21:47:01.194211 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:01.194179 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-djzzt" Jul 02 21:47:10.469218 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.469181 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs"] Jul 02 21:47:10.469827 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.469808 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51eaab2f-160d-404d-bf0c-aafccbb0654e" containerName="manager" Jul 02 21:47:10.469878 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.469830 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="51eaab2f-160d-404d-bf0c-aafccbb0654e" containerName="manager" Jul 02 21:47:10.469948 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.469938 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="51eaab2f-160d-404d-bf0c-aafccbb0654e" containerName="manager" Jul 02 21:47:10.663959 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.663929 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs"] Jul 02 21:47:10.664127 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.664029 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.666994 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.666969 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-kz6gm\"" Jul 02 21:47:10.763981 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.763897 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/619e39a7-9426-48dc-a639-12103eeaead4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.763981 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.763937 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.763981 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.763953 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619e39a7-9426-48dc-a639-12103eeaead4-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.764257 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.764018 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/619e39a7-9426-48dc-a639-12103eeaead4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.764257 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.764132 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.764257 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.764160 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.764257 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.764229 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.764453 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.764269 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.764453 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.764291 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqh5l\" (UniqueName: \"kubernetes.io/projected/619e39a7-9426-48dc-a639-12103eeaead4-kube-api-access-lqh5l\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865267 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865230 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865267 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865273 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqh5l\" (UniqueName: \"kubernetes.io/projected/619e39a7-9426-48dc-a639-12103eeaead4-kube-api-access-lqh5l\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865542 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865301 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/619e39a7-9426-48dc-a639-12103eeaead4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865542 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865323 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865542 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865338 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619e39a7-9426-48dc-a639-12103eeaead4-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865542 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865362 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/619e39a7-9426-48dc-a639-12103eeaead4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865542 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865483 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865542 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865510 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865871 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865565 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865871 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865653 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865984 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865924 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.865984 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865944 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.866062 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.865998 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.866201 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.866174 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/619e39a7-9426-48dc-a639-12103eeaead4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.867582 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.867565 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/619e39a7-9426-48dc-a639-12103eeaead4-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.867844 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.867828 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/619e39a7-9426-48dc-a639-12103eeaead4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.873759 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.873732 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqh5l\" (UniqueName: \"kubernetes.io/projected/619e39a7-9426-48dc-a639-12103eeaead4-kube-api-access-lqh5l\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.873942 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.873925 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619e39a7-9426-48dc-a639-12103eeaead4-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-q7lbs\" (UID: \"619e39a7-9426-48dc-a639-12103eeaead4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:10.974353 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:10.974325 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:11.150738 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:11.150713 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs"] Jul 02 21:47:11.152450 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:47:11.152423 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619e39a7_9426_48dc_a639_12103eeaead4.slice/crio-ab9cb333e582d944e239d3cc2aa6f510169f4db40b1af167f77c6998ee9df4b4 WatchSource:0}: Error finding container ab9cb333e582d944e239d3cc2aa6f510169f4db40b1af167f77c6998ee9df4b4: Status 404 returned error can't find the container with id ab9cb333e582d944e239d3cc2aa6f510169f4db40b1af167f77c6998ee9df4b4 Jul 02 21:47:11.154960 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:11.154919 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:47:11.155038 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:11.155009 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:47:11.155074 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:11.155057 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:47:11.324330 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:11.324285 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" event={"ID":"619e39a7-9426-48dc-a639-12103eeaead4","Type":"ContainerStarted","Data":"9ff555fb6401407943d0fbc3efe0614f9947a65965c1af0be6e06bade257f12c"} Jul 02 21:47:11.324532 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:11.324404 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" event={"ID":"619e39a7-9426-48dc-a639-12103eeaead4","Type":"ContainerStarted","Data":"ab9cb333e582d944e239d3cc2aa6f510169f4db40b1af167f77c6998ee9df4b4"} Jul 02 21:47:11.343238 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:11.343193 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" podStartSLOduration=1.343178564 podStartE2EDuration="1.343178564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:47:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:47:11.341600005 +0000 UTC m=+965.791438629" watchObservedRunningTime="2026-07-02 21:47:11.343178564 +0000 UTC m=+965.793017188" Jul 02 21:47:11.975289 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:11.975260 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:12.357974 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.357946 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-l47lz"] Jul 02 21:47:12.394468 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.394435 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-l47lz"] Jul 02 21:47:12.394620 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.394536 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:12.397007 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.396981 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 21:47:12.452569 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.452535 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-l47lz"] Jul 02 21:47:12.479222 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.479190 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nxlf\" (UniqueName: \"kubernetes.io/projected/e13bf726-4662-47f0-a281-5d4e4d99df35-kube-api-access-4nxlf\") pod \"limitador-limitador-64c94fd64f-l47lz\" (UID: \"e13bf726-4662-47f0-a281-5d4e4d99df35\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:12.479412 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.479264 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e13bf726-4662-47f0-a281-5d4e4d99df35-config-file\") pod \"limitador-limitador-64c94fd64f-l47lz\" (UID: \"e13bf726-4662-47f0-a281-5d4e4d99df35\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:12.580567 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.580537 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nxlf\" (UniqueName: \"kubernetes.io/projected/e13bf726-4662-47f0-a281-5d4e4d99df35-kube-api-access-4nxlf\") pod \"limitador-limitador-64c94fd64f-l47lz\" (UID: \"e13bf726-4662-47f0-a281-5d4e4d99df35\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:12.580752 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.580598 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e13bf726-4662-47f0-a281-5d4e4d99df35-config-file\") pod \"limitador-limitador-64c94fd64f-l47lz\" (UID: \"e13bf726-4662-47f0-a281-5d4e4d99df35\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:12.581160 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.581141 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e13bf726-4662-47f0-a281-5d4e4d99df35-config-file\") pod \"limitador-limitador-64c94fd64f-l47lz\" (UID: \"e13bf726-4662-47f0-a281-5d4e4d99df35\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:12.588746 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.588726 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nxlf\" (UniqueName: \"kubernetes.io/projected/e13bf726-4662-47f0-a281-5d4e4d99df35-kube-api-access-4nxlf\") pod \"limitador-limitador-64c94fd64f-l47lz\" (UID: \"e13bf726-4662-47f0-a281-5d4e4d99df35\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:12.705714 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.705637 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:12.859762 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.859736 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-l47lz"] Jul 02 21:47:12.861238 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:47:12.861208 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13bf726_4662_47f0_a281_5d4e4d99df35.slice/crio-e3c20ae4ea1531d36d8ff44f1471c2b308f4438519d1ed8b5f919812318d1337 WatchSource:0}: Error finding container e3c20ae4ea1531d36d8ff44f1471c2b308f4438519d1ed8b5f919812318d1337: Status 404 returned error can't find the container with id e3c20ae4ea1531d36d8ff44f1471c2b308f4438519d1ed8b5f919812318d1337 Jul 02 21:47:12.980415 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:12.980333 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:13.171940 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.171906 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mslws"] Jul 02 21:47:13.208435 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.208336 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mslws"] Jul 02 21:47:13.208611 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.208471 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mslws" Jul 02 21:47:13.211169 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.211148 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-26fc8\"" Jul 02 21:47:13.287915 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.287829 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zrm9\" (UniqueName: \"kubernetes.io/projected/4963af7a-557b-4dad-94c8-94dd0ebff946-kube-api-access-2zrm9\") pod \"authorino-58cd955675-mslws\" (UID: \"4963af7a-557b-4dad-94c8-94dd0ebff946\") " pod="kuadrant-system/authorino-58cd955675-mslws" Jul 02 21:47:13.339763 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.339709 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" event={"ID":"e13bf726-4662-47f0-a281-5d4e4d99df35","Type":"ContainerStarted","Data":"e3c20ae4ea1531d36d8ff44f1471c2b308f4438519d1ed8b5f919812318d1337"} Jul 02 21:47:13.340115 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.340091 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:13.341251 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.341232 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:47:13.388319 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.388290 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zrm9\" (UniqueName: \"kubernetes.io/projected/4963af7a-557b-4dad-94c8-94dd0ebff946-kube-api-access-2zrm9\") pod \"authorino-58cd955675-mslws\" (UID: \"4963af7a-557b-4dad-94c8-94dd0ebff946\") " pod="kuadrant-system/authorino-58cd955675-mslws" Jul 02 21:47:13.396810 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.396783 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zrm9\" (UniqueName: \"kubernetes.io/projected/4963af7a-557b-4dad-94c8-94dd0ebff946-kube-api-access-2zrm9\") pod \"authorino-58cd955675-mslws\" (UID: \"4963af7a-557b-4dad-94c8-94dd0ebff946\") " pod="kuadrant-system/authorino-58cd955675-mslws" Jul 02 21:47:13.518838 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.518801 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mslws" Jul 02 21:47:13.701324 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:13.699978 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mslws"] Jul 02 21:47:14.346612 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:14.346575 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mslws" event={"ID":"4963af7a-557b-4dad-94c8-94dd0ebff946","Type":"ContainerStarted","Data":"641bbdd8aff67d364baff44d6442778748bc1cf20d1b0f4f6b97e7fba100ccfd"} Jul 02 21:47:17.373261 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:17.373229 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" event={"ID":"e13bf726-4662-47f0-a281-5d4e4d99df35","Type":"ContainerStarted","Data":"b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3"} Jul 02 21:47:17.373836 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:17.373503 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:17.393778 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:17.393564 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" podStartSLOduration=1.3424482580000001 podStartE2EDuration="5.393546029s" totalImagesPullingTime="4.051097771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:47:17.38889009 +0000 UTC m=+971.838728713" watchObservedRunningTime="2026-07-02 21:47:17.393546029 +0000 UTC m=+971.843384655" Jul 02 21:47:19.106950 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:19.106913 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mslws"] Jul 02 21:47:19.388436 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:19.388374 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mslws" event={"ID":"4963af7a-557b-4dad-94c8-94dd0ebff946","Type":"ContainerStarted","Data":"138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0"} Jul 02 21:47:19.402732 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:19.402690 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mslws" podStartSLOduration=1.1539466 podStartE2EDuration="6.40267788s" totalImagesPullingTime="5.24873128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:47:19.401921664 +0000 UTC m=+973.851760287" watchObservedRunningTime="2026-07-02 21:47:19.40267788 +0000 UTC m=+973.852516504" Jul 02 21:47:20.395515 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:20.395454 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mslws" podUID="4963af7a-557b-4dad-94c8-94dd0ebff946" containerName="authorino" containerID="cri-o://138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0" gracePeriod=30 Jul 02 21:47:20.650880 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:20.650822 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mslws" Jul 02 21:47:20.764582 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:20.764555 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zrm9\" (UniqueName: \"kubernetes.io/projected/4963af7a-557b-4dad-94c8-94dd0ebff946-kube-api-access-2zrm9\") pod \"4963af7a-557b-4dad-94c8-94dd0ebff946\" (UID: \"4963af7a-557b-4dad-94c8-94dd0ebff946\") " Jul 02 21:47:20.766477 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:20.766450 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4963af7a-557b-4dad-94c8-94dd0ebff946-kube-api-access-2zrm9" (OuterVolumeSpecName: "kube-api-access-2zrm9") pod "4963af7a-557b-4dad-94c8-94dd0ebff946" (UID: "4963af7a-557b-4dad-94c8-94dd0ebff946"). InnerVolumeSpecName "kube-api-access-2zrm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:47:20.865886 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:20.865852 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zrm9\" (UniqueName: \"kubernetes.io/projected/4963af7a-557b-4dad-94c8-94dd0ebff946-kube-api-access-2zrm9\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:47:21.402428 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:21.402370 2547 generic.go:356] "Generic (PLEG): container finished" podID="4963af7a-557b-4dad-94c8-94dd0ebff946" containerID="138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0" exitCode=0 Jul 02 21:47:21.402428 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:21.402410 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mslws" event={"ID":"4963af7a-557b-4dad-94c8-94dd0ebff946","Type":"ContainerDied","Data":"138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0"} Jul 02 21:47:21.402923 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:21.402453 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mslws" event={"ID":"4963af7a-557b-4dad-94c8-94dd0ebff946","Type":"ContainerDied","Data":"641bbdd8aff67d364baff44d6442778748bc1cf20d1b0f4f6b97e7fba100ccfd"} Jul 02 21:47:21.402923 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:21.402451 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mslws" Jul 02 21:47:21.402923 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:21.402474 2547 scope.go:117] "RemoveContainer" containerID="138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0" Jul 02 21:47:21.412256 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:21.412240 2547 scope.go:117] "RemoveContainer" containerID="138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0" Jul 02 21:47:21.412519 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:47:21.412501 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0\": container with ID starting with 138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0 not found: ID does not exist" containerID="138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0" Jul 02 21:47:21.412608 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:21.412530 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0"} err="failed to get container status \"138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0\": rpc error: code = NotFound desc = could not find container \"138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0\": container with ID starting with 138403b3efd0d05f90d7af24ddd198cd31362f421fc70d51c650717236bb6db0 not found: ID does not exist" Jul 02 21:47:21.423748 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:21.423725 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mslws"] Jul 02 21:47:21.427590 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:21.427571 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mslws"] Jul 02 21:47:22.008852 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:22.008820 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4963af7a-557b-4dad-94c8-94dd0ebff946" path="/var/lib/kubelet/pods/4963af7a-557b-4dad-94c8-94dd0ebff946/volumes" Jul 02 21:47:28.100720 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.100628 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-l47lz"] Jul 02 21:47:28.101173 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.100872 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" podUID="e13bf726-4662-47f0-a281-5d4e4d99df35" containerName="limitador" containerID="cri-o://b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3" gracePeriod=30 Jul 02 21:47:28.101546 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.101497 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:28.662046 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.661984 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:28.733488 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.733453 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nxlf\" (UniqueName: \"kubernetes.io/projected/e13bf726-4662-47f0-a281-5d4e4d99df35-kube-api-access-4nxlf\") pod \"e13bf726-4662-47f0-a281-5d4e4d99df35\" (UID: \"e13bf726-4662-47f0-a281-5d4e4d99df35\") " Jul 02 21:47:28.733644 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.733497 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e13bf726-4662-47f0-a281-5d4e4d99df35-config-file\") pod \"e13bf726-4662-47f0-a281-5d4e4d99df35\" (UID: \"e13bf726-4662-47f0-a281-5d4e4d99df35\") " Jul 02 21:47:28.733902 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.733877 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e13bf726-4662-47f0-a281-5d4e4d99df35-config-file" (OuterVolumeSpecName: "config-file") pod "e13bf726-4662-47f0-a281-5d4e4d99df35" (UID: "e13bf726-4662-47f0-a281-5d4e4d99df35"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 21:47:28.735557 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.735533 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e13bf726-4662-47f0-a281-5d4e4d99df35-kube-api-access-4nxlf" (OuterVolumeSpecName: "kube-api-access-4nxlf") pod "e13bf726-4662-47f0-a281-5d4e4d99df35" (UID: "e13bf726-4662-47f0-a281-5d4e4d99df35"). InnerVolumeSpecName "kube-api-access-4nxlf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:47:28.834049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.834016 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nxlf\" (UniqueName: \"kubernetes.io/projected/e13bf726-4662-47f0-a281-5d4e4d99df35-kube-api-access-4nxlf\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:47:28.834049 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:28.834043 2547 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e13bf726-4662-47f0-a281-5d4e4d99df35-config-file\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:47:29.450459 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:29.450420 2547 generic.go:356] "Generic (PLEG): container finished" podID="e13bf726-4662-47f0-a281-5d4e4d99df35" containerID="b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3" exitCode=0 Jul 02 21:47:29.450847 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:29.450471 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" Jul 02 21:47:29.450847 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:29.450466 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" event={"ID":"e13bf726-4662-47f0-a281-5d4e4d99df35","Type":"ContainerDied","Data":"b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3"} Jul 02 21:47:29.450847 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:29.450597 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-l47lz" event={"ID":"e13bf726-4662-47f0-a281-5d4e4d99df35","Type":"ContainerDied","Data":"e3c20ae4ea1531d36d8ff44f1471c2b308f4438519d1ed8b5f919812318d1337"} Jul 02 21:47:29.450847 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:29.450628 2547 scope.go:117] "RemoveContainer" containerID="b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3" Jul 02 21:47:29.460135 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:29.460113 2547 scope.go:117] "RemoveContainer" containerID="b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3" Jul 02 21:47:29.460416 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:47:29.460395 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3\": container with ID starting with b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3 not found: ID does not exist" containerID="b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3" Jul 02 21:47:29.460493 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:29.460423 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3"} err="failed to get container status \"b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3\": rpc error: code = NotFound desc = could not find container \"b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3\": container with ID starting with b3cf81ed3ed3d492b3619bd07fde9038350298df8f20cdabe134a555e338a2d3 not found: ID does not exist" Jul 02 21:47:29.472742 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:29.472717 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-l47lz"] Jul 02 21:47:29.476417 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:29.476397 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-l47lz"] Jul 02 21:47:30.008917 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:30.008883 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e13bf726-4662-47f0-a281-5d4e4d99df35" path="/var/lib/kubelet/pods/e13bf726-4662-47f0-a281-5d4e4d99df35/volumes" Jul 02 21:47:33.995105 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:33.995072 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-88fd2"] Jul 02 21:47:33.995730 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:33.995695 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e13bf726-4662-47f0-a281-5d4e4d99df35" containerName="limitador" Jul 02 21:47:33.995730 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:33.995722 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e13bf726-4662-47f0-a281-5d4e4d99df35" containerName="limitador" Jul 02 21:47:33.995906 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:33.995741 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4963af7a-557b-4dad-94c8-94dd0ebff946" containerName="authorino" Jul 02 21:47:33.995906 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:33.995748 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="4963af7a-557b-4dad-94c8-94dd0ebff946" containerName="authorino" Jul 02 21:47:33.995906 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:33.995833 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="4963af7a-557b-4dad-94c8-94dd0ebff946" containerName="authorino" Jul 02 21:47:33.995906 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:33.995843 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e13bf726-4662-47f0-a281-5d4e4d99df35" containerName="limitador" Jul 02 21:47:33.999039 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:33.999022 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:34.001565 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.001540 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-djnps\"" Jul 02 21:47:34.001674 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.001540 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 02 21:47:34.012680 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.012657 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-88fd2"] Jul 02 21:47:34.070816 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.070784 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhs5v\" (UniqueName: \"kubernetes.io/projected/a548cb99-e8e8-4d1d-a196-35fbef85d57a-kube-api-access-mhs5v\") pod \"postgres-69fb4594cf-88fd2\" (UID: \"a548cb99-e8e8-4d1d-a196-35fbef85d57a\") " pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:34.070816 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.070818 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a548cb99-e8e8-4d1d-a196-35fbef85d57a-data\") pod \"postgres-69fb4594cf-88fd2\" (UID: \"a548cb99-e8e8-4d1d-a196-35fbef85d57a\") " pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:34.171700 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.171665 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhs5v\" (UniqueName: \"kubernetes.io/projected/a548cb99-e8e8-4d1d-a196-35fbef85d57a-kube-api-access-mhs5v\") pod \"postgres-69fb4594cf-88fd2\" (UID: \"a548cb99-e8e8-4d1d-a196-35fbef85d57a\") " pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:34.171700 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.171701 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a548cb99-e8e8-4d1d-a196-35fbef85d57a-data\") pod \"postgres-69fb4594cf-88fd2\" (UID: \"a548cb99-e8e8-4d1d-a196-35fbef85d57a\") " pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:34.172039 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.172022 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a548cb99-e8e8-4d1d-a196-35fbef85d57a-data\") pod \"postgres-69fb4594cf-88fd2\" (UID: \"a548cb99-e8e8-4d1d-a196-35fbef85d57a\") " pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:34.179437 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.179410 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhs5v\" (UniqueName: \"kubernetes.io/projected/a548cb99-e8e8-4d1d-a196-35fbef85d57a-kube-api-access-mhs5v\") pod \"postgres-69fb4594cf-88fd2\" (UID: \"a548cb99-e8e8-4d1d-a196-35fbef85d57a\") " pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:34.313702 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.313671 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:34.446936 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.446908 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-88fd2"] Jul 02 21:47:34.448097 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:47:34.448073 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda548cb99_e8e8_4d1d_a196_35fbef85d57a.slice/crio-17efe122c06a8afe47920751bae47225ad9eb5a5ddcadf8e130f076be645eacc WatchSource:0}: Error finding container 17efe122c06a8afe47920751bae47225ad9eb5a5ddcadf8e130f076be645eacc: Status 404 returned error can't find the container with id 17efe122c06a8afe47920751bae47225ad9eb5a5ddcadf8e130f076be645eacc Jul 02 21:47:34.479425 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:34.479397 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-88fd2" event={"ID":"a548cb99-e8e8-4d1d-a196-35fbef85d57a","Type":"ContainerStarted","Data":"17efe122c06a8afe47920751bae47225ad9eb5a5ddcadf8e130f076be645eacc"} Jul 02 21:47:39.521183 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:39.521087 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-88fd2" event={"ID":"a548cb99-e8e8-4d1d-a196-35fbef85d57a","Type":"ContainerStarted","Data":"b3b209d1a5481b6c2cd508d88b955893a7995bedcc90cbbc7191bae9f9e6ac24"} Jul 02 21:47:39.521668 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:39.521235 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:39.537572 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:39.537513 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-88fd2" podStartSLOduration=1.800527791 podStartE2EDuration="6.53749806s" totalImagesPullingTime="4.736970269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:47:39.535926843 +0000 UTC m=+993.985765466" watchObservedRunningTime="2026-07-02 21:47:39.53749806 +0000 UTC m=+993.987336684" Jul 02 21:47:45.556211 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:45.556183 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-88fd2" Jul 02 21:47:46.106608 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.106573 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-df5dj"] Jul 02 21:47:46.113231 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.113210 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-df5dj" Jul 02 21:47:46.115547 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.115525 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-26fc8\"" Jul 02 21:47:46.117050 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.117024 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-df5dj"] Jul 02 21:47:46.187792 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.187758 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdxbr\" (UniqueName: \"kubernetes.io/projected/39d7ad7e-dc4c-4858-8966-383c6708a80e-kube-api-access-fdxbr\") pod \"authorino-5d4b67c446-df5dj\" (UID: \"39d7ad7e-dc4c-4858-8966-383c6708a80e\") " pod="kuadrant-system/authorino-5d4b67c446-df5dj" Jul 02 21:47:46.276232 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.276193 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-df5dj"] Jul 02 21:47:46.276532 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:47:46.276510 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-fdxbr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-df5dj" podUID="39d7ad7e-dc4c-4858-8966-383c6708a80e" Jul 02 21:47:46.288561 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.288534 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdxbr\" (UniqueName: \"kubernetes.io/projected/39d7ad7e-dc4c-4858-8966-383c6708a80e-kube-api-access-fdxbr\") pod \"authorino-5d4b67c446-df5dj\" (UID: \"39d7ad7e-dc4c-4858-8966-383c6708a80e\") " pod="kuadrant-system/authorino-5d4b67c446-df5dj" Jul 02 21:47:46.299337 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.299299 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdxbr\" (UniqueName: \"kubernetes.io/projected/39d7ad7e-dc4c-4858-8966-383c6708a80e-kube-api-access-fdxbr\") pod \"authorino-5d4b67c446-df5dj\" (UID: \"39d7ad7e-dc4c-4858-8966-383c6708a80e\") " pod="kuadrant-system/authorino-5d4b67c446-df5dj" Jul 02 21:47:46.305201 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.305177 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-9xcjp"] Jul 02 21:47:46.309512 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.309494 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:46.311699 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.311681 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 02 21:47:46.316167 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.316146 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-9xcjp"] Jul 02 21:47:46.363565 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.363481 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-9xcjp"] Jul 02 21:47:46.363819 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:47:46.363795 2547 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-zdqwt tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-9xcjp" podUID="9afab3ca-8a00-48ee-9733-1567ab9a8c15" Jul 02 21:47:46.385657 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.385625 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-694fb888f8-ppnrr"] Jul 02 21:47:46.389438 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.389401 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9afab3ca-8a00-48ee-9733-1567ab9a8c15-tls-cert\") pod \"authorino-76877994-9xcjp\" (UID: \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\") " pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:46.389590 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.389489 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdqwt\" (UniqueName: \"kubernetes.io/projected/9afab3ca-8a00-48ee-9733-1567ab9a8c15-kube-api-access-zdqwt\") pod \"authorino-76877994-9xcjp\" (UID: \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\") " pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:46.389944 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.389929 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:47:46.396743 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.396715 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-694fb888f8-ppnrr"] Jul 02 21:47:46.490463 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.490428 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdqwt\" (UniqueName: \"kubernetes.io/projected/9afab3ca-8a00-48ee-9733-1567ab9a8c15-kube-api-access-zdqwt\") pod \"authorino-76877994-9xcjp\" (UID: \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\") " pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:46.490643 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.490484 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88baa583-5a71-4e11-b215-14e007e53051-tls-cert\") pod \"authorino-694fb888f8-ppnrr\" (UID: \"88baa583-5a71-4e11-b215-14e007e53051\") " pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:47:46.490643 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.490510 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5nhh\" (UniqueName: \"kubernetes.io/projected/88baa583-5a71-4e11-b215-14e007e53051-kube-api-access-c5nhh\") pod \"authorino-694fb888f8-ppnrr\" (UID: \"88baa583-5a71-4e11-b215-14e007e53051\") " pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:47:46.490643 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.490542 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9afab3ca-8a00-48ee-9733-1567ab9a8c15-tls-cert\") pod \"authorino-76877994-9xcjp\" (UID: \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\") " pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:46.492845 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.492827 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9afab3ca-8a00-48ee-9733-1567ab9a8c15-tls-cert\") pod \"authorino-76877994-9xcjp\" (UID: \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\") " pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:46.498104 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.498080 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdqwt\" (UniqueName: \"kubernetes.io/projected/9afab3ca-8a00-48ee-9733-1567ab9a8c15-kube-api-access-zdqwt\") pod \"authorino-76877994-9xcjp\" (UID: \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\") " pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:46.557708 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.557683 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-df5dj" Jul 02 21:47:46.558054 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.557682 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:46.565593 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.565573 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-df5dj" Jul 02 21:47:46.568714 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.568695 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:46.591625 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.591607 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88baa583-5a71-4e11-b215-14e007e53051-tls-cert\") pod \"authorino-694fb888f8-ppnrr\" (UID: \"88baa583-5a71-4e11-b215-14e007e53051\") " pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:47:46.591740 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.591637 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5nhh\" (UniqueName: \"kubernetes.io/projected/88baa583-5a71-4e11-b215-14e007e53051-kube-api-access-c5nhh\") pod \"authorino-694fb888f8-ppnrr\" (UID: \"88baa583-5a71-4e11-b215-14e007e53051\") " pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:47:46.593987 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.593967 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88baa583-5a71-4e11-b215-14e007e53051-tls-cert\") pod \"authorino-694fb888f8-ppnrr\" (UID: \"88baa583-5a71-4e11-b215-14e007e53051\") " pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:47:46.598815 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.598794 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5nhh\" (UniqueName: \"kubernetes.io/projected/88baa583-5a71-4e11-b215-14e007e53051-kube-api-access-c5nhh\") pod \"authorino-694fb888f8-ppnrr\" (UID: \"88baa583-5a71-4e11-b215-14e007e53051\") " pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:47:46.692541 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.692454 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9afab3ca-8a00-48ee-9733-1567ab9a8c15-tls-cert\") pod \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\" (UID: \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\") " Jul 02 21:47:46.692706 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.692563 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdqwt\" (UniqueName: \"kubernetes.io/projected/9afab3ca-8a00-48ee-9733-1567ab9a8c15-kube-api-access-zdqwt\") pod \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\" (UID: \"9afab3ca-8a00-48ee-9733-1567ab9a8c15\") " Jul 02 21:47:46.692706 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.692620 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdxbr\" (UniqueName: \"kubernetes.io/projected/39d7ad7e-dc4c-4858-8966-383c6708a80e-kube-api-access-fdxbr\") pod \"39d7ad7e-dc4c-4858-8966-383c6708a80e\" (UID: \"39d7ad7e-dc4c-4858-8966-383c6708a80e\") " Jul 02 21:47:46.694575 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.694544 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9afab3ca-8a00-48ee-9733-1567ab9a8c15-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9afab3ca-8a00-48ee-9733-1567ab9a8c15" (UID: "9afab3ca-8a00-48ee-9733-1567ab9a8c15"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:47:46.694575 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.694561 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9afab3ca-8a00-48ee-9733-1567ab9a8c15-kube-api-access-zdqwt" (OuterVolumeSpecName: "kube-api-access-zdqwt") pod "9afab3ca-8a00-48ee-9733-1567ab9a8c15" (UID: "9afab3ca-8a00-48ee-9733-1567ab9a8c15"). InnerVolumeSpecName "kube-api-access-zdqwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:47:46.694718 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.694606 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39d7ad7e-dc4c-4858-8966-383c6708a80e-kube-api-access-fdxbr" (OuterVolumeSpecName: "kube-api-access-fdxbr") pod "39d7ad7e-dc4c-4858-8966-383c6708a80e" (UID: "39d7ad7e-dc4c-4858-8966-383c6708a80e"). InnerVolumeSpecName "kube-api-access-fdxbr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:47:46.699772 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.699753 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:47:46.794302 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.794265 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdxbr\" (UniqueName: \"kubernetes.io/projected/39d7ad7e-dc4c-4858-8966-383c6708a80e-kube-api-access-fdxbr\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:47:46.794445 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.794304 2547 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9afab3ca-8a00-48ee-9733-1567ab9a8c15-tls-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:47:46.794445 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.794322 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdqwt\" (UniqueName: \"kubernetes.io/projected/9afab3ca-8a00-48ee-9733-1567ab9a8c15-kube-api-access-zdqwt\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:47:46.837264 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:46.837237 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-694fb888f8-ppnrr"] Jul 02 21:47:46.838699 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:47:46.838671 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88baa583_5a71_4e11_b215_14e007e53051.slice/crio-e31cb432174f31218727d115b3061a5af23d10ec84b8fb14a4ff79c06be46b2c WatchSource:0}: Error finding container e31cb432174f31218727d115b3061a5af23d10ec84b8fb14a4ff79c06be46b2c: Status 404 returned error can't find the container with id e31cb432174f31218727d115b3061a5af23d10ec84b8fb14a4ff79c06be46b2c Jul 02 21:47:47.565088 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:47.565050 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-694fb888f8-ppnrr" event={"ID":"88baa583-5a71-4e11-b215-14e007e53051","Type":"ContainerStarted","Data":"ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2"} Jul 02 21:47:47.565088 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:47.565093 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-694fb888f8-ppnrr" event={"ID":"88baa583-5a71-4e11-b215-14e007e53051","Type":"ContainerStarted","Data":"e31cb432174f31218727d115b3061a5af23d10ec84b8fb14a4ff79c06be46b2c"} Jul 02 21:47:47.565582 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:47.565101 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-9xcjp" Jul 02 21:47:47.565582 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:47.565119 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-df5dj" Jul 02 21:47:47.583939 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:47.583899 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-694fb888f8-ppnrr" podStartSLOduration=1.227836177 podStartE2EDuration="1.583883614s" totalImagesPullingTime="356.047437ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:47:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:47:47.583538116 +0000 UTC m=+1002.033376761" watchObservedRunningTime="2026-07-02 21:47:47.583883614 +0000 UTC m=+1002.033722237" Jul 02 21:47:47.609226 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:47.609195 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-9xcjp"] Jul 02 21:47:47.612032 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:47.612006 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-9xcjp"] Jul 02 21:47:47.641441 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:47.641408 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-df5dj"] Jul 02 21:47:47.643587 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:47.643564 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-df5dj"] Jul 02 21:47:48.009503 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:48.009425 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39d7ad7e-dc4c-4858-8966-383c6708a80e" path="/var/lib/kubelet/pods/39d7ad7e-dc4c-4858-8966-383c6708a80e/volumes" Jul 02 21:47:48.009700 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:48.009687 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9afab3ca-8a00-48ee-9733-1567ab9a8c15" path="/var/lib/kubelet/pods/9afab3ca-8a00-48ee-9733-1567ab9a8c15/volumes" Jul 02 21:47:52.185269 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.185236 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75fcc9d99-stlsk"] Jul 02 21:47:52.257251 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.257212 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75fcc9d99-stlsk"] Jul 02 21:47:52.257436 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.257340 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:47:52.260278 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.260255 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 02 21:47:52.260442 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.260263 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 02 21:47:52.260442 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.260265 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wcc5k\"" Jul 02 21:47:52.345542 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.345508 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56gsm\" (UniqueName: \"kubernetes.io/projected/9a0fdded-7e6c-41bc-a252-36a40843ea78-kube-api-access-56gsm\") pod \"maas-controller-75fcc9d99-stlsk\" (UID: \"9a0fdded-7e6c-41bc-a252-36a40843ea78\") " pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:47:52.345719 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.345566 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a0fdded-7e6c-41bc-a252-36a40843ea78-webhook-cert\") pod \"maas-controller-75fcc9d99-stlsk\" (UID: \"9a0fdded-7e6c-41bc-a252-36a40843ea78\") " pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:47:52.350705 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.350681 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-794b6f76fd-6fz5f"] Jul 02 21:47:52.371240 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.371213 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794b6f76fd-6fz5f"] Jul 02 21:47:52.371396 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.371331 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:47:52.446998 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.446908 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a0fdded-7e6c-41bc-a252-36a40843ea78-webhook-cert\") pod \"maas-controller-75fcc9d99-stlsk\" (UID: \"9a0fdded-7e6c-41bc-a252-36a40843ea78\") " pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:47:52.447184 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.447000 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcm2p\" (UniqueName: \"kubernetes.io/projected/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-kube-api-access-wcm2p\") pod \"maas-controller-794b6f76fd-6fz5f\" (UID: \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\") " pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:47:52.447184 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.447071 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56gsm\" (UniqueName: \"kubernetes.io/projected/9a0fdded-7e6c-41bc-a252-36a40843ea78-kube-api-access-56gsm\") pod \"maas-controller-75fcc9d99-stlsk\" (UID: \"9a0fdded-7e6c-41bc-a252-36a40843ea78\") " pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:47:52.447184 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.447142 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-webhook-cert\") pod \"maas-controller-794b6f76fd-6fz5f\" (UID: \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\") " pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:47:52.449470 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.449449 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a0fdded-7e6c-41bc-a252-36a40843ea78-webhook-cert\") pod \"maas-controller-75fcc9d99-stlsk\" (UID: \"9a0fdded-7e6c-41bc-a252-36a40843ea78\") " pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:47:52.455990 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.455966 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56gsm\" (UniqueName: \"kubernetes.io/projected/9a0fdded-7e6c-41bc-a252-36a40843ea78-kube-api-access-56gsm\") pod \"maas-controller-75fcc9d99-stlsk\" (UID: \"9a0fdded-7e6c-41bc-a252-36a40843ea78\") " pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:47:52.548129 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.548091 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-webhook-cert\") pod \"maas-controller-794b6f76fd-6fz5f\" (UID: \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\") " pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:47:52.548285 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.548165 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcm2p\" (UniqueName: \"kubernetes.io/projected/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-kube-api-access-wcm2p\") pod \"maas-controller-794b6f76fd-6fz5f\" (UID: \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\") " pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:47:52.550506 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.550478 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-webhook-cert\") pod \"maas-controller-794b6f76fd-6fz5f\" (UID: \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\") " pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:47:52.556713 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.556689 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcm2p\" (UniqueName: \"kubernetes.io/projected/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-kube-api-access-wcm2p\") pod \"maas-controller-794b6f76fd-6fz5f\" (UID: \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\") " pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:47:52.567512 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.567492 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:47:52.682492 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.682461 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:47:52.742686 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.742665 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75fcc9d99-stlsk"] Jul 02 21:47:52.745067 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:47:52.744990 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a0fdded_7e6c_41bc_a252_36a40843ea78.slice/crio-bce70475225b7d4fbe5f81f3e8393d1a825fa23b53496ab5ebe6331597c37763 WatchSource:0}: Error finding container bce70475225b7d4fbe5f81f3e8393d1a825fa23b53496ab5ebe6331597c37763: Status 404 returned error can't find the container with id bce70475225b7d4fbe5f81f3e8393d1a825fa23b53496ab5ebe6331597c37763 Jul 02 21:47:52.839322 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:52.839298 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794b6f76fd-6fz5f"] Jul 02 21:47:52.841016 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:47:52.840978 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1e711b_9a6c_492b_a94a_e3cf8d0253fe.slice/crio-1445028531de04a26a54edc65213fc80e5753c62b6ac55288e0f7e692cb54cf1 WatchSource:0}: Error finding container 1445028531de04a26a54edc65213fc80e5753c62b6ac55288e0f7e692cb54cf1: Status 404 returned error can't find the container with id 1445028531de04a26a54edc65213fc80e5753c62b6ac55288e0f7e692cb54cf1 Jul 02 21:47:53.602315 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:53.602255 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75fcc9d99-stlsk" event={"ID":"9a0fdded-7e6c-41bc-a252-36a40843ea78","Type":"ContainerStarted","Data":"bce70475225b7d4fbe5f81f3e8393d1a825fa23b53496ab5ebe6331597c37763"} Jul 02 21:47:53.603798 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:53.603737 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" event={"ID":"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe","Type":"ContainerStarted","Data":"1445028531de04a26a54edc65213fc80e5753c62b6ac55288e0f7e692cb54cf1"} Jul 02 21:47:55.618756 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:55.618717 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75fcc9d99-stlsk" event={"ID":"9a0fdded-7e6c-41bc-a252-36a40843ea78","Type":"ContainerStarted","Data":"6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0"} Jul 02 21:47:55.619220 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:55.618849 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:47:55.620497 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:55.620473 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" event={"ID":"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe","Type":"ContainerStarted","Data":"da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a"} Jul 02 21:47:55.620600 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:55.620531 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:47:55.641068 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:55.641029 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75fcc9d99-stlsk" podStartSLOduration=1.129556207 podStartE2EDuration="3.641016107s" totalImagesPullingTime="2.5114599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:47:55.638860334 +0000 UTC m=+1010.088698957" watchObservedRunningTime="2026-07-02 21:47:55.641016107 +0000 UTC m=+1010.090854727" Jul 02 21:47:55.663917 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:47:55.663864 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" podStartSLOduration=1.236688251 podStartE2EDuration="3.663849933s" totalImagesPullingTime="2.427161682s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:47:55.661909999 +0000 UTC m=+1010.111748646" watchObservedRunningTime="2026-07-02 21:47:55.663849933 +0000 UTC m=+1010.113688558" Jul 02 21:48:06.631761 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:06.631732 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:48:06.632245 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:06.631813 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:48:06.692909 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:06.692879 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75fcc9d99-stlsk"] Jul 02 21:48:06.693115 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:06.693067 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-75fcc9d99-stlsk" podUID="9a0fdded-7e6c-41bc-a252-36a40843ea78" containerName="manager" containerID="cri-o://6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0" gracePeriod=30 Jul 02 21:48:06.943826 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:06.943803 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:48:07.088833 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.088801 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a0fdded-7e6c-41bc-a252-36a40843ea78-webhook-cert\") pod \"9a0fdded-7e6c-41bc-a252-36a40843ea78\" (UID: \"9a0fdded-7e6c-41bc-a252-36a40843ea78\") " Jul 02 21:48:07.089042 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.088903 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56gsm\" (UniqueName: \"kubernetes.io/projected/9a0fdded-7e6c-41bc-a252-36a40843ea78-kube-api-access-56gsm\") pod \"9a0fdded-7e6c-41bc-a252-36a40843ea78\" (UID: \"9a0fdded-7e6c-41bc-a252-36a40843ea78\") " Jul 02 21:48:07.091006 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.090979 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a0fdded-7e6c-41bc-a252-36a40843ea78-kube-api-access-56gsm" (OuterVolumeSpecName: "kube-api-access-56gsm") pod "9a0fdded-7e6c-41bc-a252-36a40843ea78" (UID: "9a0fdded-7e6c-41bc-a252-36a40843ea78"). InnerVolumeSpecName "kube-api-access-56gsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:48:07.091107 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.091008 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a0fdded-7e6c-41bc-a252-36a40843ea78-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9a0fdded-7e6c-41bc-a252-36a40843ea78" (UID: "9a0fdded-7e6c-41bc-a252-36a40843ea78"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:48:07.190063 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.189975 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56gsm\" (UniqueName: \"kubernetes.io/projected/9a0fdded-7e6c-41bc-a252-36a40843ea78-kube-api-access-56gsm\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:48:07.190063 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.190002 2547 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a0fdded-7e6c-41bc-a252-36a40843ea78-webhook-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:48:07.696112 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.696080 2547 generic.go:356] "Generic (PLEG): container finished" podID="9a0fdded-7e6c-41bc-a252-36a40843ea78" containerID="6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0" exitCode=0 Jul 02 21:48:07.696688 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.696123 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75fcc9d99-stlsk" Jul 02 21:48:07.696688 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.696169 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75fcc9d99-stlsk" event={"ID":"9a0fdded-7e6c-41bc-a252-36a40843ea78","Type":"ContainerDied","Data":"6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0"} Jul 02 21:48:07.696688 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.696224 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75fcc9d99-stlsk" event={"ID":"9a0fdded-7e6c-41bc-a252-36a40843ea78","Type":"ContainerDied","Data":"bce70475225b7d4fbe5f81f3e8393d1a825fa23b53496ab5ebe6331597c37763"} Jul 02 21:48:07.696688 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.696259 2547 scope.go:117] "RemoveContainer" containerID="6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0" Jul 02 21:48:07.705778 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.705763 2547 scope.go:117] "RemoveContainer" containerID="6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0" Jul 02 21:48:07.706025 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:48:07.706007 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0\": container with ID starting with 6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0 not found: ID does not exist" containerID="6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0" Jul 02 21:48:07.706093 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.706033 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0"} err="failed to get container status \"6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0\": rpc error: code = NotFound desc = could not find container \"6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0\": container with ID starting with 6c769e414c724e4de4bdc68ccfd445efcbcbddb0cb9b40b0c589756a283067b0 not found: ID does not exist" Jul 02 21:48:07.717564 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.717543 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75fcc9d99-stlsk"] Jul 02 21:48:07.721243 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:07.721224 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75fcc9d99-stlsk"] Jul 02 21:48:08.009469 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:08.009361 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a0fdded-7e6c-41bc-a252-36a40843ea78" path="/var/lib/kubelet/pods/9a0fdded-7e6c-41bc-a252-36a40843ea78/volumes" Jul 02 21:48:27.373579 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.373546 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-gwgmr"] Jul 02 21:48:27.374068 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.374007 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a0fdded-7e6c-41bc-a252-36a40843ea78" containerName="manager" Jul 02 21:48:27.374068 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.374019 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a0fdded-7e6c-41bc-a252-36a40843ea78" containerName="manager" Jul 02 21:48:27.374146 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.374089 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a0fdded-7e6c-41bc-a252-36a40843ea78" containerName="manager" Jul 02 21:48:27.380068 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.380013 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:27.382473 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.382447 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 02 21:48:27.382612 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.382480 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-t9wzx\"" Jul 02 21:48:27.386903 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.386880 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-gwgmr"] Jul 02 21:48:27.419851 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.419817 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8b9x9"] Jul 02 21:48:27.424549 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.424530 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:27.434311 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.434284 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8b9x9"] Jul 02 21:48:27.483728 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.483688 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmzxt\" (UniqueName: \"kubernetes.io/projected/5730229c-9c39-4ecf-84d8-2d4579577ca3-kube-api-access-bmzxt\") pod \"payload-pre-processing-7964457696-8b9x9\" (UID: \"5730229c-9c39-4ecf-84d8-2d4579577ca3\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:27.483903 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.483800 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5730229c-9c39-4ecf-84d8-2d4579577ca3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8b9x9\" (UID: \"5730229c-9c39-4ecf-84d8-2d4579577ca3\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:27.483903 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.483827 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a392b5e-9307-4bc5-971c-48848af0a307-plugins-config-volume\") pod \"payload-processing-58f86c7798-gwgmr\" (UID: \"2a392b5e-9307-4bc5-971c-48848af0a307\") " pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:27.483903 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.483855 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ptlh\" (UniqueName: \"kubernetes.io/projected/2a392b5e-9307-4bc5-971c-48848af0a307-kube-api-access-7ptlh\") pod \"payload-processing-58f86c7798-gwgmr\" (UID: \"2a392b5e-9307-4bc5-971c-48848af0a307\") " pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:27.585394 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.585358 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmzxt\" (UniqueName: \"kubernetes.io/projected/5730229c-9c39-4ecf-84d8-2d4579577ca3-kube-api-access-bmzxt\") pod \"payload-pre-processing-7964457696-8b9x9\" (UID: \"5730229c-9c39-4ecf-84d8-2d4579577ca3\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:27.585586 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.585483 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5730229c-9c39-4ecf-84d8-2d4579577ca3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8b9x9\" (UID: \"5730229c-9c39-4ecf-84d8-2d4579577ca3\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:27.585586 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.585515 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a392b5e-9307-4bc5-971c-48848af0a307-plugins-config-volume\") pod \"payload-processing-58f86c7798-gwgmr\" (UID: \"2a392b5e-9307-4bc5-971c-48848af0a307\") " pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:27.585586 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.585538 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ptlh\" (UniqueName: \"kubernetes.io/projected/2a392b5e-9307-4bc5-971c-48848af0a307-kube-api-access-7ptlh\") pod \"payload-processing-58f86c7798-gwgmr\" (UID: \"2a392b5e-9307-4bc5-971c-48848af0a307\") " pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:27.586293 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.586259 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5730229c-9c39-4ecf-84d8-2d4579577ca3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-8b9x9\" (UID: \"5730229c-9c39-4ecf-84d8-2d4579577ca3\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:27.586428 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.586321 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a392b5e-9307-4bc5-971c-48848af0a307-plugins-config-volume\") pod \"payload-processing-58f86c7798-gwgmr\" (UID: \"2a392b5e-9307-4bc5-971c-48848af0a307\") " pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:27.593651 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.593628 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmzxt\" (UniqueName: \"kubernetes.io/projected/5730229c-9c39-4ecf-84d8-2d4579577ca3-kube-api-access-bmzxt\") pod \"payload-pre-processing-7964457696-8b9x9\" (UID: \"5730229c-9c39-4ecf-84d8-2d4579577ca3\") " pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:27.593759 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.593728 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ptlh\" (UniqueName: \"kubernetes.io/projected/2a392b5e-9307-4bc5-971c-48848af0a307-kube-api-access-7ptlh\") pod \"payload-processing-58f86c7798-gwgmr\" (UID: \"2a392b5e-9307-4bc5-971c-48848af0a307\") " pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:27.692872 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.692773 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:27.736629 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.736583 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:27.857117 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.857026 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-gwgmr"] Jul 02 21:48:27.859508 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:48:27.859478 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a392b5e_9307_4bc5_971c_48848af0a307.slice/crio-93bc4f197199d9f01c6060c6f1b76b12bd1822226040d80d973e3e24c731cc4b WatchSource:0}: Error finding container 93bc4f197199d9f01c6060c6f1b76b12bd1822226040d80d973e3e24c731cc4b: Status 404 returned error can't find the container with id 93bc4f197199d9f01c6060c6f1b76b12bd1822226040d80d973e3e24c731cc4b Jul 02 21:48:27.862083 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.862066 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 21:48:27.901374 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:27.901340 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-8b9x9"] Jul 02 21:48:27.901860 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:48:27.901833 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5730229c_9c39_4ecf_84d8_2d4579577ca3.slice/crio-ac513cee87b43764970e2a723952b3c971da95f3aa88b1a08d5023ce10bf0e7f WatchSource:0}: Error finding container ac513cee87b43764970e2a723952b3c971da95f3aa88b1a08d5023ce10bf0e7f: Status 404 returned error can't find the container with id ac513cee87b43764970e2a723952b3c971da95f3aa88b1a08d5023ce10bf0e7f Jul 02 21:48:28.824716 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:28.824646 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" event={"ID":"5730229c-9c39-4ecf-84d8-2d4579577ca3","Type":"ContainerStarted","Data":"ac513cee87b43764970e2a723952b3c971da95f3aa88b1a08d5023ce10bf0e7f"} Jul 02 21:48:28.826540 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:28.826407 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" event={"ID":"2a392b5e-9307-4bc5-971c-48848af0a307","Type":"ContainerStarted","Data":"93bc4f197199d9f01c6060c6f1b76b12bd1822226040d80d973e3e24c731cc4b"} Jul 02 21:48:29.837738 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:29.837703 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" event={"ID":"5730229c-9c39-4ecf-84d8-2d4579577ca3","Type":"ContainerStarted","Data":"4dfa3704d5ce5e94d58771a23062733ee5c28971ce031776e32742bd0b682f8f"} Jul 02 21:48:29.838186 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:29.837764 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:29.839240 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:29.839219 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" event={"ID":"2a392b5e-9307-4bc5-971c-48848af0a307","Type":"ContainerStarted","Data":"a70faeb7719958789db91325f14a611ef029a6b77e3227898d79198f153d6328"} Jul 02 21:48:29.839335 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:29.839268 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:29.856863 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:29.856822 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" podStartSLOduration=1.225786582 podStartE2EDuration="2.856807441s" totalImagesPullingTime="1.631020859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:48:29.854556999 +0000 UTC m=+1044.304395623" watchObservedRunningTime="2026-07-02 21:48:29.856807441 +0000 UTC m=+1044.306646065" Jul 02 21:48:29.870490 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:29.870437 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" podStartSLOduration=1.195202728 podStartE2EDuration="2.870416672s" totalImagesPullingTime="1.675213944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:48:29.869366199 +0000 UTC m=+1044.319204823" watchObservedRunningTime="2026-07-02 21:48:29.870416672 +0000 UTC m=+1044.320255298" Jul 02 21:48:35.876704 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:35.876667 2547 generic.go:356] "Generic (PLEG): container finished" podID="619e39a7-9426-48dc-a639-12103eeaead4" containerID="9ff555fb6401407943d0fbc3efe0614f9947a65965c1af0be6e06bade257f12c" exitCode=0 Jul 02 21:48:35.877149 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:35.876744 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" event={"ID":"619e39a7-9426-48dc-a639-12103eeaead4","Type":"ContainerDied","Data":"9ff555fb6401407943d0fbc3efe0614f9947a65965c1af0be6e06bade257f12c"} Jul 02 21:48:35.877204 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:35.877147 2547 scope.go:117] "RemoveContainer" containerID="9ff555fb6401407943d0fbc3efe0614f9947a65965c1af0be6e06bade257f12c" Jul 02 21:48:35.878134 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:35.878109 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:48:35.878190 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:35.878181 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:48:35.878237 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:35.878215 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 21:48:36.902194 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:36.902158 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" event={"ID":"619e39a7-9426-48dc-a639-12103eeaead4","Type":"ContainerStarted","Data":"e91d01fdce2450e5db2a184efda9bca20e25cd9fcfed756995efd7756d566de7"} Jul 02 21:48:36.984870 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:36.984824 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:48:36.985100 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:36.984955 2547 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-q7lbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 02 21:48:36.985100 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:36.985007 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" podUID="619e39a7-9426-48dc-a639-12103eeaead4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 02 21:48:37.974981 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:37.974944 2547 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-q7lbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 02 21:48:37.975580 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:37.975547 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" podUID="619e39a7-9426-48dc-a639-12103eeaead4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 02 21:48:38.975526 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:38.975055 2547 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-q7lbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 02 21:48:38.975526 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:38.975131 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" podUID="619e39a7-9426-48dc-a639-12103eeaead4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 02 21:48:39.974796 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:39.974759 2547 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-q7lbs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 02 21:48:39.975116 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:39.975086 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" podUID="619e39a7-9426-48dc-a639-12103eeaead4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 02 21:48:40.847238 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:40.847199 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-gwgmr" Jul 02 21:48:40.847713 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:40.847514 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-8b9x9" Jul 02 21:48:40.978747 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:40.978715 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:48:40.979021 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:40.979002 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:48:40.979612 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:40.979592 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-q7lbs" Jul 02 21:48:52.140188 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.140145 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw"] Jul 02 21:48:52.145470 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.145440 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.147984 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.147963 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 21:48:52.149057 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.149026 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p7grx\"" Jul 02 21:48:52.149156 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.149099 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 21:48:52.149156 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.149098 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 02 21:48:52.153680 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.153653 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw"] Jul 02 21:48:52.232870 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.232832 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6a07706-b120-4b5c-b997-ae518bcf61c8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.232870 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.232876 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.233100 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.232910 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.233100 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.232936 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.233100 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.232973 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f52l2\" (UniqueName: \"kubernetes.io/projected/b6a07706-b120-4b5c-b997-ae518bcf61c8-kube-api-access-f52l2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.233100 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.233050 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.334203 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.334172 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.334426 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.334211 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.334426 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.334227 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.334426 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.334249 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f52l2\" (UniqueName: \"kubernetes.io/projected/b6a07706-b120-4b5c-b997-ae518bcf61c8-kube-api-access-f52l2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.334426 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.334299 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.334668 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.334447 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6a07706-b120-4b5c-b997-ae518bcf61c8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.334668 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.334597 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.334668 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.334618 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.334841 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.334691 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.336522 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.336503 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b6a07706-b120-4b5c-b997-ae518bcf61c8-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.336917 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.336897 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b6a07706-b120-4b5c-b997-ae518bcf61c8-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.342136 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.342114 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f52l2\" (UniqueName: \"kubernetes.io/projected/b6a07706-b120-4b5c-b997-ae518bcf61c8-kube-api-access-f52l2\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw\" (UID: \"b6a07706-b120-4b5c-b997-ae518bcf61c8\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.457419 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.457302 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:48:52.623030 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:52.623003 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw"] Jul 02 21:48:52.623318 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:48:52.623295 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6a07706_b120_4b5c_b997_ae518bcf61c8.slice/crio-045e88c4fd6ed4c7d3eef35f8bfa00e43dfb34c12277dcaa194e39cbedc0d082 WatchSource:0}: Error finding container 045e88c4fd6ed4c7d3eef35f8bfa00e43dfb34c12277dcaa194e39cbedc0d082: Status 404 returned error can't find the container with id 045e88c4fd6ed4c7d3eef35f8bfa00e43dfb34c12277dcaa194e39cbedc0d082 Jul 02 21:48:53.006168 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:53.006122 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" event={"ID":"b6a07706-b120-4b5c-b997-ae518bcf61c8","Type":"ContainerStarted","Data":"045e88c4fd6ed4c7d3eef35f8bfa00e43dfb34c12277dcaa194e39cbedc0d082"} Jul 02 21:48:54.013521 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:54.013425 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" event={"ID":"b6a07706-b120-4b5c-b997-ae518bcf61c8","Type":"ContainerStarted","Data":"c95dce1229c56fd04037c5b1bc196bf387417c57f44800d2ad2733e0a19733d5"} Jul 02 21:48:54.030682 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:48:54.030621 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" podStartSLOduration=0.947674852 podStartE2EDuration="2.030600689s" totalImagesPullingTime="1.082925837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:48:54.030020073 +0000 UTC m=+1068.479858697" watchObservedRunningTime="2026-07-02 21:48:54.030600689 +0000 UTC m=+1068.480439315" Jul 02 21:49:02.458025 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:02.457989 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:49:02.458624 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:02.458074 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:49:02.471486 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:02.471460 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:49:03.076541 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:03.076510 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw" Jul 02 21:49:20.831353 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.831313 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8"] Jul 02 21:49:20.836840 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.836816 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:20.839893 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.839868 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 02 21:49:20.845634 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.845606 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8"] Jul 02 21:49:20.909767 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.909727 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:20.909956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.909830 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:20.909956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.909870 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87b80306-b4d9-4a24-b1a1-c99a9137ce86-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:20.909956 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.909951 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:20.910111 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.909977 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:20.910111 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:20.910004 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fspj4\" (UniqueName: \"kubernetes.io/projected/87b80306-b4d9-4a24-b1a1-c99a9137ce86-kube-api-access-fspj4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.011350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.011307 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.011350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.011350 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.011627 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.011418 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fspj4\" (UniqueName: \"kubernetes.io/projected/87b80306-b4d9-4a24-b1a1-c99a9137ce86-kube-api-access-fspj4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.011627 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.011464 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.011627 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.011553 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.011627 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.011600 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87b80306-b4d9-4a24-b1a1-c99a9137ce86-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.011845 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.011819 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.011943 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.011920 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.011983 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.011952 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.013851 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.013821 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87b80306-b4d9-4a24-b1a1-c99a9137ce86-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.014075 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.014057 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87b80306-b4d9-4a24-b1a1-c99a9137ce86-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.019054 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.019033 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fspj4\" (UniqueName: \"kubernetes.io/projected/87b80306-b4d9-4a24-b1a1-c99a9137ce86-kube-api-access-fspj4\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8\" (UID: \"87b80306-b4d9-4a24-b1a1-c99a9137ce86\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.148037 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.147951 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:21.288620 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:21.288592 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8"] Jul 02 21:49:21.289713 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:49:21.289689 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87b80306_b4d9_4a24_b1a1_c99a9137ce86.slice/crio-25dbfa21a16fa1e87249bf12cc729a243eda722b969898c4b26e2bf9d2214bba WatchSource:0}: Error finding container 25dbfa21a16fa1e87249bf12cc729a243eda722b969898c4b26e2bf9d2214bba: Status 404 returned error can't find the container with id 25dbfa21a16fa1e87249bf12cc729a243eda722b969898c4b26e2bf9d2214bba Jul 02 21:49:22.178026 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:22.177993 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" event={"ID":"87b80306-b4d9-4a24-b1a1-c99a9137ce86","Type":"ContainerStarted","Data":"6cbef070dc1d9f86c1f78f58c4a390905afd22eca4b7f1d04e45755ff5ee8999"} Jul 02 21:49:22.178519 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:22.178051 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" event={"ID":"87b80306-b4d9-4a24-b1a1-c99a9137ce86","Type":"ContainerStarted","Data":"25dbfa21a16fa1e87249bf12cc729a243eda722b969898c4b26e2bf9d2214bba"} Jul 02 21:49:22.195135 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:22.195071 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" podStartSLOduration=2.011464035 podStartE2EDuration="2.195050188s" totalImagesPullingTime="183.586153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:49:22.194090773 +0000 UTC m=+1096.643929397" watchObservedRunningTime="2026-07-02 21:49:22.195050188 +0000 UTC m=+1096.644888817" Jul 02 21:49:31.148558 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:31.148515 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:31.149085 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:31.148601 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:31.161629 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:31.161599 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:49:31.243517 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:49:31.243487 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8" Jul 02 21:50:44.951699 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:44.951605 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-694fb888f8-ppnrr"] Jul 02 21:50:44.952251 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:44.951840 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-694fb888f8-ppnrr" podUID="88baa583-5a71-4e11-b215-14e007e53051" containerName="authorino" containerID="cri-o://ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2" gracePeriod=30 Jul 02 21:50:45.219529 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.219500 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:50:45.281194 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.281154 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5nhh\" (UniqueName: \"kubernetes.io/projected/88baa583-5a71-4e11-b215-14e007e53051-kube-api-access-c5nhh\") pod \"88baa583-5a71-4e11-b215-14e007e53051\" (UID: \"88baa583-5a71-4e11-b215-14e007e53051\") " Jul 02 21:50:45.281355 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.281205 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88baa583-5a71-4e11-b215-14e007e53051-tls-cert\") pod \"88baa583-5a71-4e11-b215-14e007e53051\" (UID: \"88baa583-5a71-4e11-b215-14e007e53051\") " Jul 02 21:50:45.283243 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.283216 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88baa583-5a71-4e11-b215-14e007e53051-kube-api-access-c5nhh" (OuterVolumeSpecName: "kube-api-access-c5nhh") pod "88baa583-5a71-4e11-b215-14e007e53051" (UID: "88baa583-5a71-4e11-b215-14e007e53051"). InnerVolumeSpecName "kube-api-access-c5nhh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:50:45.292408 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.292353 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88baa583-5a71-4e11-b215-14e007e53051-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "88baa583-5a71-4e11-b215-14e007e53051" (UID: "88baa583-5a71-4e11-b215-14e007e53051"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:50:45.382292 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.382262 2547 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/88baa583-5a71-4e11-b215-14e007e53051-tls-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:50:45.382292 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.382288 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5nhh\" (UniqueName: \"kubernetes.io/projected/88baa583-5a71-4e11-b215-14e007e53051-kube-api-access-c5nhh\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:50:45.672422 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.672371 2547 generic.go:356] "Generic (PLEG): container finished" podID="88baa583-5a71-4e11-b215-14e007e53051" containerID="ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2" exitCode=0 Jul 02 21:50:45.672422 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.672406 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-694fb888f8-ppnrr" event={"ID":"88baa583-5a71-4e11-b215-14e007e53051","Type":"ContainerDied","Data":"ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2"} Jul 02 21:50:45.672422 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.672438 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-694fb888f8-ppnrr" Jul 02 21:50:45.672727 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.672454 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-694fb888f8-ppnrr" event={"ID":"88baa583-5a71-4e11-b215-14e007e53051","Type":"ContainerDied","Data":"e31cb432174f31218727d115b3061a5af23d10ec84b8fb14a4ff79c06be46b2c"} Jul 02 21:50:45.672727 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.672475 2547 scope.go:117] "RemoveContainer" containerID="ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2" Jul 02 21:50:45.682290 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.682269 2547 scope.go:117] "RemoveContainer" containerID="ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2" Jul 02 21:50:45.682580 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:50:45.682564 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2\": container with ID starting with ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2 not found: ID does not exist" containerID="ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2" Jul 02 21:50:45.682639 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.682587 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2"} err="failed to get container status \"ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2\": rpc error: code = NotFound desc = could not find container \"ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2\": container with ID starting with ccefec9789ba0ac6ad671c265541db768e54e8058d19a67ac7f3a6bd517403d2 not found: ID does not exist" Jul 02 21:50:45.694473 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.694440 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-694fb888f8-ppnrr"] Jul 02 21:50:45.698565 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:45.698541 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-694fb888f8-ppnrr"] Jul 02 21:50:46.009609 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:50:46.009524 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88baa583-5a71-4e11-b215-14e007e53051" path="/var/lib/kubelet/pods/88baa583-5a71-4e11-b215-14e007e53051/volumes" Jul 02 21:51:05.975976 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:05.975936 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:51:05.976968 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:05.976949 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:51:05.983693 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:05.983662 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:51:05.985149 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:05.985122 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:51:59.347539 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:59.347435 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-794b6f76fd-6fz5f"] Jul 02 21:51:59.348095 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:59.347729 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" podUID="2d1e711b-9a6c-492b-a94a-e3cf8d0253fe" containerName="manager" containerID="cri-o://da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a" gracePeriod=30 Jul 02 21:51:59.610789 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:59.610724 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:51:59.780154 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:59.780124 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-webhook-cert\") pod \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\" (UID: \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\") " Jul 02 21:51:59.780325 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:59.780225 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcm2p\" (UniqueName: \"kubernetes.io/projected/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-kube-api-access-wcm2p\") pod \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\" (UID: \"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe\") " Jul 02 21:51:59.782304 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:59.782267 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2d1e711b-9a6c-492b-a94a-e3cf8d0253fe" (UID: "2d1e711b-9a6c-492b-a94a-e3cf8d0253fe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 21:51:59.782304 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:59.782289 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-kube-api-access-wcm2p" (OuterVolumeSpecName: "kube-api-access-wcm2p") pod "2d1e711b-9a6c-492b-a94a-e3cf8d0253fe" (UID: "2d1e711b-9a6c-492b-a94a-e3cf8d0253fe"). InnerVolumeSpecName "kube-api-access-wcm2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 21:51:59.881080 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:59.880986 2547 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-webhook-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:51:59.881080 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:51:59.881015 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcm2p\" (UniqueName: \"kubernetes.io/projected/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe-kube-api-access-wcm2p\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 21:52:00.114926 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.114885 2547 generic.go:356] "Generic (PLEG): container finished" podID="2d1e711b-9a6c-492b-a94a-e3cf8d0253fe" containerID="da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a" exitCode=0 Jul 02 21:52:00.114926 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.114930 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" Jul 02 21:52:00.115144 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.114956 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" event={"ID":"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe","Type":"ContainerDied","Data":"da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a"} Jul 02 21:52:00.115144 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.115005 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b6f76fd-6fz5f" event={"ID":"2d1e711b-9a6c-492b-a94a-e3cf8d0253fe","Type":"ContainerDied","Data":"1445028531de04a26a54edc65213fc80e5753c62b6ac55288e0f7e692cb54cf1"} Jul 02 21:52:00.115144 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.115028 2547 scope.go:117] "RemoveContainer" containerID="da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a" Jul 02 21:52:00.124334 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.124296 2547 scope.go:117] "RemoveContainer" containerID="da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a" Jul 02 21:52:00.124602 ip-10-0-137-35 kubenswrapper[2547]: E0702 21:52:00.124582 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a\": container with ID starting with da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a not found: ID does not exist" containerID="da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a" Jul 02 21:52:00.124660 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.124611 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a"} err="failed to get container status \"da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a\": rpc error: code = NotFound desc = could not find container \"da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a\": container with ID starting with da7649398655880a322ed69febabe4deaf1a13f1c453b563eb38895eb69e5a3a not found: ID does not exist" Jul 02 21:52:00.133531 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.133456 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-794b6f76fd-6fz5f"] Jul 02 21:52:00.137599 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.137570 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-794b6f76fd-6fz5f"] Jul 02 21:52:00.921279 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.921241 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-794b6f76fd-ttmvw"] Jul 02 21:52:00.921819 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.921798 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1e711b-9a6c-492b-a94a-e3cf8d0253fe" containerName="manager" Jul 02 21:52:00.921819 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.921818 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1e711b-9a6c-492b-a94a-e3cf8d0253fe" containerName="manager" Jul 02 21:52:00.921970 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.921840 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88baa583-5a71-4e11-b215-14e007e53051" containerName="authorino" Jul 02 21:52:00.921970 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.921846 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="88baa583-5a71-4e11-b215-14e007e53051" containerName="authorino" Jul 02 21:52:00.921970 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.921922 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1e711b-9a6c-492b-a94a-e3cf8d0253fe" containerName="manager" Jul 02 21:52:00.921970 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.921932 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="88baa583-5a71-4e11-b215-14e007e53051" containerName="authorino" Jul 02 21:52:00.926265 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.926249 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:52:00.929850 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.929829 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 02 21:52:00.929945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.929832 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wcc5k\"" Jul 02 21:52:00.929945 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.929832 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 02 21:52:00.935464 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:00.935430 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794b6f76fd-ttmvw"] Jul 02 21:52:01.093639 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:01.093598 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hkjp\" (UniqueName: \"kubernetes.io/projected/1a074673-6489-4874-b9e5-723117d22873-kube-api-access-6hkjp\") pod \"maas-controller-794b6f76fd-ttmvw\" (UID: \"1a074673-6489-4874-b9e5-723117d22873\") " pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:52:01.093844 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:01.093662 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a074673-6489-4874-b9e5-723117d22873-webhook-cert\") pod \"maas-controller-794b6f76fd-ttmvw\" (UID: \"1a074673-6489-4874-b9e5-723117d22873\") " pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:52:01.194567 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:01.194488 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a074673-6489-4874-b9e5-723117d22873-webhook-cert\") pod \"maas-controller-794b6f76fd-ttmvw\" (UID: \"1a074673-6489-4874-b9e5-723117d22873\") " pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:52:01.194721 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:01.194585 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hkjp\" (UniqueName: \"kubernetes.io/projected/1a074673-6489-4874-b9e5-723117d22873-kube-api-access-6hkjp\") pod \"maas-controller-794b6f76fd-ttmvw\" (UID: \"1a074673-6489-4874-b9e5-723117d22873\") " pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:52:01.196852 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:01.196833 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a074673-6489-4874-b9e5-723117d22873-webhook-cert\") pod \"maas-controller-794b6f76fd-ttmvw\" (UID: \"1a074673-6489-4874-b9e5-723117d22873\") " pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:52:01.206885 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:01.206857 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hkjp\" (UniqueName: \"kubernetes.io/projected/1a074673-6489-4874-b9e5-723117d22873-kube-api-access-6hkjp\") pod \"maas-controller-794b6f76fd-ttmvw\" (UID: \"1a074673-6489-4874-b9e5-723117d22873\") " pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:52:01.237350 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:01.237328 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:52:01.384632 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:01.384600 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794b6f76fd-ttmvw"] Jul 02 21:52:01.385790 ip-10-0-137-35 kubenswrapper[2547]: W0702 21:52:01.385758 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a074673_6489_4874_b9e5_723117d22873.slice/crio-75d6c1d19cf29f3b59b3348690e4f933e2668f0bd8b71f345754e0877fbeda4c WatchSource:0}: Error finding container 75d6c1d19cf29f3b59b3348690e4f933e2668f0bd8b71f345754e0877fbeda4c: Status 404 returned error can't find the container with id 75d6c1d19cf29f3b59b3348690e4f933e2668f0bd8b71f345754e0877fbeda4c Jul 02 21:52:02.016700 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:02.016670 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d1e711b-9a6c-492b-a94a-e3cf8d0253fe" path="/var/lib/kubelet/pods/2d1e711b-9a6c-492b-a94a-e3cf8d0253fe/volumes" Jul 02 21:52:02.130414 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:02.130313 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b6f76fd-ttmvw" event={"ID":"1a074673-6489-4874-b9e5-723117d22873","Type":"ContainerStarted","Data":"04d5583a8a051caaa96f4dabacedb6fa781e6775842d8951f7361c7f0249c2c6"} Jul 02 21:52:02.130414 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:02.130374 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b6f76fd-ttmvw" event={"ID":"1a074673-6489-4874-b9e5-723117d22873","Type":"ContainerStarted","Data":"75d6c1d19cf29f3b59b3348690e4f933e2668f0bd8b71f345754e0877fbeda4c"} Jul 02 21:52:02.130590 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:02.130469 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:52:02.148584 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:02.148537 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-794b6f76fd-ttmvw" podStartSLOduration=1.663468257 podStartE2EDuration="2.148519938s" totalImagesPullingTime="485.051681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 21:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 21:52:02.147056173 +0000 UTC m=+1256.596894822" watchObservedRunningTime="2026-07-02 21:52:02.148519938 +0000 UTC m=+1256.598358561" Jul 02 21:52:13.142503 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:52:13.142470 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-794b6f76fd-ttmvw" Jul 02 21:56:06.025892 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:56:06.025864 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:56:06.028340 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:56:06.028311 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 21:56:06.034036 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:56:06.034014 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 21:56:06.036188 ip-10-0-137-35 kubenswrapper[2547]: I0702 21:56:06.036171 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 22:00:00.147619 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.147587 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717160-2b244"] Jul 02 22:00:00.151297 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.151277 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717160-2b244" Jul 02 22:00:00.154083 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.154051 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-vnrlr\"" Jul 02 22:00:00.168232 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.168200 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717160-2b244"] Jul 02 22:00:00.336779 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.336742 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g26kn\" (UniqueName: \"kubernetes.io/projected/ea3b61bb-b6a8-4bad-a25a-031c89bab0a2-kube-api-access-g26kn\") pod \"maas-api-key-cleanup-29717160-2b244\" (UID: \"ea3b61bb-b6a8-4bad-a25a-031c89bab0a2\") " pod="opendatahub/maas-api-key-cleanup-29717160-2b244" Jul 02 22:00:00.437780 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.437678 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g26kn\" (UniqueName: \"kubernetes.io/projected/ea3b61bb-b6a8-4bad-a25a-031c89bab0a2-kube-api-access-g26kn\") pod \"maas-api-key-cleanup-29717160-2b244\" (UID: \"ea3b61bb-b6a8-4bad-a25a-031c89bab0a2\") " pod="opendatahub/maas-api-key-cleanup-29717160-2b244" Jul 02 22:00:00.447142 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.447118 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g26kn\" (UniqueName: \"kubernetes.io/projected/ea3b61bb-b6a8-4bad-a25a-031c89bab0a2-kube-api-access-g26kn\") pod \"maas-api-key-cleanup-29717160-2b244\" (UID: \"ea3b61bb-b6a8-4bad-a25a-031c89bab0a2\") " pod="opendatahub/maas-api-key-cleanup-29717160-2b244" Jul 02 22:00:00.462042 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.462015 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717160-2b244" Jul 02 22:00:00.610460 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.610432 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717160-2b244"] Jul 02 22:00:00.612522 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:00:00.612493 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3b61bb_b6a8_4bad_a25a_031c89bab0a2.slice/crio-aee487a3a7d36e493b6c851708e54a15d1f194e52e44f6a9ddfb30a96194c937 WatchSource:0}: Error finding container aee487a3a7d36e493b6c851708e54a15d1f194e52e44f6a9ddfb30a96194c937: Status 404 returned error can't find the container with id aee487a3a7d36e493b6c851708e54a15d1f194e52e44f6a9ddfb30a96194c937 Jul 02 22:00:00.614755 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.614734 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 22:00:00.944446 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:00.944411 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717160-2b244" event={"ID":"ea3b61bb-b6a8-4bad-a25a-031c89bab0a2","Type":"ContainerStarted","Data":"aee487a3a7d36e493b6c851708e54a15d1f194e52e44f6a9ddfb30a96194c937"} Jul 02 22:00:06.982894 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:06.982794 2547 generic.go:356] "Generic (PLEG): container finished" podID="ea3b61bb-b6a8-4bad-a25a-031c89bab0a2" containerID="651d9fc58559db5b48e122be6b249156ebce6ae3d29c1fe4e420a1e006a267bf" exitCode=0 Jul 02 22:00:06.982894 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:06.982881 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717160-2b244" event={"ID":"ea3b61bb-b6a8-4bad-a25a-031c89bab0a2","Type":"ContainerDied","Data":"651d9fc58559db5b48e122be6b249156ebce6ae3d29c1fe4e420a1e006a267bf"} Jul 02 22:00:08.132741 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:08.132713 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717160-2b244" Jul 02 22:00:08.201231 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:08.201197 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g26kn\" (UniqueName: \"kubernetes.io/projected/ea3b61bb-b6a8-4bad-a25a-031c89bab0a2-kube-api-access-g26kn\") pod \"ea3b61bb-b6a8-4bad-a25a-031c89bab0a2\" (UID: \"ea3b61bb-b6a8-4bad-a25a-031c89bab0a2\") " Jul 02 22:00:08.203233 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:08.203208 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea3b61bb-b6a8-4bad-a25a-031c89bab0a2-kube-api-access-g26kn" (OuterVolumeSpecName: "kube-api-access-g26kn") pod "ea3b61bb-b6a8-4bad-a25a-031c89bab0a2" (UID: "ea3b61bb-b6a8-4bad-a25a-031c89bab0a2"). InnerVolumeSpecName "kube-api-access-g26kn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:00:08.302223 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:08.302184 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g26kn\" (UniqueName: \"kubernetes.io/projected/ea3b61bb-b6a8-4bad-a25a-031c89bab0a2-kube-api-access-g26kn\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:00:08.995886 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:08.995859 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717160-2b244" Jul 02 22:00:08.996083 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:08.995899 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717160-2b244" event={"ID":"ea3b61bb-b6a8-4bad-a25a-031c89bab0a2","Type":"ContainerDied","Data":"aee487a3a7d36e493b6c851708e54a15d1f194e52e44f6a9ddfb30a96194c937"} Jul 02 22:00:08.996083 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:00:08.995953 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aee487a3a7d36e493b6c851708e54a15d1f194e52e44f6a9ddfb30a96194c937" Jul 02 22:01:06.065785 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:01:06.065756 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 22:01:06.070302 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:01:06.070278 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 22:01:06.073555 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:01:06.073535 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 22:01:06.077772 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:01:06.077755 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 22:04:06.596724 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.596693 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8"] Jul 02 22:04:06.597914 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.597619 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3b61bb-b6a8-4bad-a25a-031c89bab0a2" containerName="cleanup" Jul 02 22:04:06.597914 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.597642 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3b61bb-b6a8-4bad-a25a-031c89bab0a2" containerName="cleanup" Jul 02 22:04:06.597914 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.597807 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea3b61bb-b6a8-4bad-a25a-031c89bab0a2" containerName="cleanup" Jul 02 22:04:06.601956 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.601937 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.605563 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.605541 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-21dcdf63-openshift-default-dockercfg-4z64f\"" Jul 02 22:04:06.635314 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.635277 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8"] Jul 02 22:04:06.638357 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.638332 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-socket\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.638493 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.638398 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-certs\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.638493 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.638428 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-envoy\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.638615 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.638490 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-data\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.638615 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.638521 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-credential-socket\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.638615 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.638592 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-token\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.638615 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.638612 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch78v\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-kube-api-access-ch78v\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.638747 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.638644 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istiod-ca-cert\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.638747 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.638668 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-podinfo\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740187 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740148 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-socket\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740368 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740201 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-certs\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740368 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740220 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-envoy\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740368 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740254 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-data\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740368 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740275 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-credential-socket\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740368 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740331 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-token\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740368 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740356 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch78v\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-kube-api-access-ch78v\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740721 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740430 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istiod-ca-cert\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740721 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740634 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-podinfo\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740721 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740681 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-certs\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740721 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740699 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-socket\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740891 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740757 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-data\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.740891 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.740837 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-credential-socket\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.741317 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.741293 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istiod-ca-cert\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.742636 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.742613 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-envoy\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.743038 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.743018 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-podinfo\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.749265 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.749244 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch78v\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-kube-api-access-ch78v\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.749634 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.749617 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-token\") pod \"e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:06.913018 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:06.912933 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:07.061914 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:07.061887 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8"] Jul 02 22:04:07.063636 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:04:07.063604 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee3f1c2_8ea1_4203_8648_09ef92cea2bb.slice/crio-2facbdee9a353636f323a3e56f3d9c7a71ab5c391a24121825115a9b058fb6fa WatchSource:0}: Error finding container 2facbdee9a353636f323a3e56f3d9c7a71ab5c391a24121825115a9b058fb6fa: Status 404 returned error can't find the container with id 2facbdee9a353636f323a3e56f3d9c7a71ab5c391a24121825115a9b058fb6fa Jul 02 22:04:07.065904 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:07.065868 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:07.066015 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:07.065934 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:07.066015 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:07.065961 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:07.413356 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:07.413319 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" event={"ID":"aee3f1c2-8ea1-4203-8648-09ef92cea2bb","Type":"ContainerStarted","Data":"d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc"} Jul 02 22:04:07.413544 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:07.413407 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" event={"ID":"aee3f1c2-8ea1-4203-8648-09ef92cea2bb","Type":"ContainerStarted","Data":"2facbdee9a353636f323a3e56f3d9c7a71ab5c391a24121825115a9b058fb6fa"} Jul 02 22:04:07.438215 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:07.438173 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" podStartSLOduration=1.438158225 podStartE2EDuration="1.438158225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:07.435459423 +0000 UTC m=+1981.885298047" watchObservedRunningTime="2026-07-02 22:04:07.438158225 +0000 UTC m=+1981.887996848" Jul 02 22:04:07.913321 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:07.913277 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:08.918996 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:08.918960 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:09.425701 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:09.425668 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:09.426717 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:09.426699 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:13.250161 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:13.250127 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8"] Jul 02 22:04:13.250709 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:13.250301 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" podUID="aee3f1c2-8ea1-4203-8648-09ef92cea2bb" containerName="istio-proxy" containerID="cri-o://d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc" gracePeriod=30 Jul 02 22:04:19.220495 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.220441 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:19.248418 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.247250 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6"] Jul 02 22:04:19.248418 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.247956 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee3f1c2-8ea1-4203-8648-09ef92cea2bb" containerName="istio-proxy" Jul 02 22:04:19.248418 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.247975 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee3f1c2-8ea1-4203-8648-09ef92cea2bb" containerName="istio-proxy" Jul 02 22:04:19.248418 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.248075 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee3f1c2-8ea1-4203-8648-09ef92cea2bb" containerName="istio-proxy" Jul 02 22:04:19.251294 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.251270 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.254250 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.254228 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0956ab05-openshift-default-dockercfg-fslnv\"" Jul 02 22:04:19.267715 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.267683 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-podinfo\") pod \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " Jul 02 22:04:19.267834 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.267740 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-credential-socket\") pod \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " Jul 02 22:04:19.267834 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.267792 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-data\") pod \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " Jul 02 22:04:19.267957 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.267867 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-socket\") pod \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " Jul 02 22:04:19.267957 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.267902 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-envoy\") pod \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " Jul 02 22:04:19.267957 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.267948 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ch78v\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-kube-api-access-ch78v\") pod \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " Jul 02 22:04:19.268103 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.267986 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-certs\") pod \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " Jul 02 22:04:19.268103 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.268041 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-token\") pod \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " Jul 02 22:04:19.268103 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.268081 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istiod-ca-cert\") pod \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\" (UID: \"aee3f1c2-8ea1-4203-8648-09ef92cea2bb\") " Jul 02 22:04:19.268251 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.268035 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aee3f1c2-8ea1-4203-8648-09ef92cea2bb" (UID: "aee3f1c2-8ea1-4203-8648-09ef92cea2bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:19.268251 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.268196 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aee3f1c2-8ea1-4203-8648-09ef92cea2bb" (UID: "aee3f1c2-8ea1-4203-8648-09ef92cea2bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:19.268251 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.268214 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aee3f1c2-8ea1-4203-8648-09ef92cea2bb" (UID: "aee3f1c2-8ea1-4203-8648-09ef92cea2bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:19.268468 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.268449 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:19.268540 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.268473 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:19.268540 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.268491 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:19.268638 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.268533 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aee3f1c2-8ea1-4203-8648-09ef92cea2bb" (UID: "aee3f1c2-8ea1-4203-8648-09ef92cea2bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:04:19.269449 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.269426 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-data" (OuterVolumeSpecName: "istio-data") pod "aee3f1c2-8ea1-4203-8648-09ef92cea2bb" (UID: "aee3f1c2-8ea1-4203-8648-09ef92cea2bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:19.271417 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.271353 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aee3f1c2-8ea1-4203-8648-09ef92cea2bb" (UID: "aee3f1c2-8ea1-4203-8648-09ef92cea2bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:04:19.271831 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.271795 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aee3f1c2-8ea1-4203-8648-09ef92cea2bb" (UID: "aee3f1c2-8ea1-4203-8648-09ef92cea2bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:19.271945 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.271814 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-token" (OuterVolumeSpecName: "istio-token") pod "aee3f1c2-8ea1-4203-8648-09ef92cea2bb" (UID: "aee3f1c2-8ea1-4203-8648-09ef92cea2bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:04:19.272290 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.272255 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-kube-api-access-ch78v" (OuterVolumeSpecName: "kube-api-access-ch78v") pod "aee3f1c2-8ea1-4203-8648-09ef92cea2bb" (UID: "aee3f1c2-8ea1-4203-8648-09ef92cea2bb"). InnerVolumeSpecName "kube-api-access-ch78v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:04:19.275786 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.275761 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6"] Jul 02 22:04:19.370307 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370267 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istiod-ca-cert\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.370528 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370333 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-certs\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.370528 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370409 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-data\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.370528 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370468 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqxfn\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-kube-api-access-wqxfn\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.370664 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370565 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-envoy\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.370664 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370597 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-credential-socket\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.370664 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370621 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-token\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.370772 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370665 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-socket\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.370772 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370682 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-podinfo\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.370772 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370725 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:19.370772 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370735 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:19.370772 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370744 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:19.370772 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370751 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:19.370772 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370760 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:19.370772 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.370769 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch78v\" (UniqueName: \"kubernetes.io/projected/aee3f1c2-8ea1-4203-8648-09ef92cea2bb-kube-api-access-ch78v\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:19.471919 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.471879 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-socket\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.471919 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.471917 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-podinfo\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472112 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472055 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istiod-ca-cert\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472112 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472105 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-certs\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472185 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472136 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-data\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472185 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472158 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqxfn\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-kube-api-access-wqxfn\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472281 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472207 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-envoy\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472281 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472253 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-credential-socket\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472281 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472277 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-token\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472479 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472409 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-socket\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472669 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472637 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-data\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472819 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472667 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-certs\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472896 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472815 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-credential-socket\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.472896 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.472827 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istiod-ca-cert\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.474472 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.474445 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-envoy\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.474472 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.474464 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-podinfo\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.485737 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.485710 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-token\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.485947 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.485930 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqxfn\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-kube-api-access-wqxfn\") pod \"e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.503285 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.503261 2547 generic.go:356] "Generic (PLEG): container finished" podID="aee3f1c2-8ea1-4203-8648-09ef92cea2bb" containerID="d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc" exitCode=0 Jul 02 22:04:19.503412 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.503314 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" Jul 02 22:04:19.503412 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.503346 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" event={"ID":"aee3f1c2-8ea1-4203-8648-09ef92cea2bb","Type":"ContainerDied","Data":"d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc"} Jul 02 22:04:19.503547 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.503416 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8" event={"ID":"aee3f1c2-8ea1-4203-8648-09ef92cea2bb","Type":"ContainerDied","Data":"2facbdee9a353636f323a3e56f3d9c7a71ab5c391a24121825115a9b058fb6fa"} Jul 02 22:04:19.503547 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.503443 2547 scope.go:117] "RemoveContainer" containerID="d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc" Jul 02 22:04:19.522079 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.522062 2547 scope.go:117] "RemoveContainer" containerID="d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc" Jul 02 22:04:19.522362 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:04:19.522340 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc\": container with ID starting with d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc not found: ID does not exist" containerID="d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc" Jul 02 22:04:19.522497 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.522368 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc"} err="failed to get container status \"d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc\": rpc error: code = NotFound desc = could not find container \"d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc\": container with ID starting with d5e8c8a8a42ad30d00148f083a31060de23fb371d44bdd7a50122d9c2783d8bc not found: ID does not exist" Jul 02 22:04:19.542730 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.542713 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8"] Jul 02 22:04:19.552349 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.552331 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-21dcdf63-openshift-default-69f55ff957-dmnv8"] Jul 02 22:04:19.563564 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.563548 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:19.703065 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.703038 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6"] Jul 02 22:04:19.704675 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:04:19.704648 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323246b7_6b69_407e_ad2d_f5b5dd49ac1b.slice/crio-cbd11c657f8bb41145288c4ebe4eaa70f5f9ebac9c0ab28dd68e836be03ab831 WatchSource:0}: Error finding container cbd11c657f8bb41145288c4ebe4eaa70f5f9ebac9c0ab28dd68e836be03ab831: Status 404 returned error can't find the container with id cbd11c657f8bb41145288c4ebe4eaa70f5f9ebac9c0ab28dd68e836be03ab831 Jul 02 22:04:19.706770 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.706736 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:19.706849 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.706824 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:19.706895 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:19.706857 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:20.009911 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:20.009827 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aee3f1c2-8ea1-4203-8648-09ef92cea2bb" path="/var/lib/kubelet/pods/aee3f1c2-8ea1-4203-8648-09ef92cea2bb/volumes" Jul 02 22:04:20.514787 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:20.514748 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" event={"ID":"323246b7-6b69-407e-ad2d-f5b5dd49ac1b","Type":"ContainerStarted","Data":"970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355"} Jul 02 22:04:20.515293 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:20.514809 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" event={"ID":"323246b7-6b69-407e-ad2d-f5b5dd49ac1b","Type":"ContainerStarted","Data":"cbd11c657f8bb41145288c4ebe4eaa70f5f9ebac9c0ab28dd68e836be03ab831"} Jul 02 22:04:20.540301 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:20.540241 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" podStartSLOduration=1.540220082 podStartE2EDuration="1.540220082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:20.538622021 +0000 UTC m=+1994.988460638" watchObservedRunningTime="2026-07-02 22:04:20.540220082 +0000 UTC m=+1994.990058708" Jul 02 22:04:20.564239 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:20.564207 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:20.568711 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:20.568688 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:21.520744 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:21.520711 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:21.521737 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:21.521718 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:31.589735 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:31.589691 2547 generic.go:356] "Generic (PLEG): container finished" podID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" containerID="970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355" exitCode=0 Jul 02 22:04:31.590158 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:31.589765 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" event={"ID":"323246b7-6b69-407e-ad2d-f5b5dd49ac1b","Type":"ContainerDied","Data":"970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355"} Jul 02 22:04:31.590261 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:31.590248 2547 scope.go:117] "RemoveContainer" containerID="970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355" Jul 02 22:04:31.591306 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:31.591276 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:31.591421 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:31.591342 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:31.591421 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:31.591374 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:32.011156 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:32.011068 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6"] Jul 02 22:04:32.598341 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:32.598305 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" event={"ID":"323246b7-6b69-407e-ad2d-f5b5dd49ac1b","Type":"ContainerStarted","Data":"8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e"} Jul 02 22:04:32.598832 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:32.598367 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" podUID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" containerName="istio-proxy" containerID="cri-o://8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e" gracePeriod=30 Jul 02 22:04:36.521067 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:36.521034 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:37.848800 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.848773 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:37.958338 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.958263 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqxfn\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-kube-api-access-wqxfn\") pod \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " Jul 02 22:04:37.958338 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.958328 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-certs\") pod \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " Jul 02 22:04:37.958565 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.958356 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-socket\") pod \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " Jul 02 22:04:37.958565 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.958425 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istiod-ca-cert\") pod \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " Jul 02 22:04:37.958565 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.958446 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-data\") pod \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " Jul 02 22:04:37.958565 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.958478 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-credential-socket\") pod \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " Jul 02 22:04:37.958565 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.958511 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-podinfo\") pod \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " Jul 02 22:04:37.958565 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.958542 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-envoy\") pod \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " Jul 02 22:04:37.958753 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.958588 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-token\") pod \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\" (UID: \"323246b7-6b69-407e-ad2d-f5b5dd49ac1b\") " Jul 02 22:04:37.961581 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.961555 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "323246b7-6b69-407e-ad2d-f5b5dd49ac1b" (UID: "323246b7-6b69-407e-ad2d-f5b5dd49ac1b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:37.961722 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.961620 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "323246b7-6b69-407e-ad2d-f5b5dd49ac1b" (UID: "323246b7-6b69-407e-ad2d-f5b5dd49ac1b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:37.961802 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.961723 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "323246b7-6b69-407e-ad2d-f5b5dd49ac1b" (UID: "323246b7-6b69-407e-ad2d-f5b5dd49ac1b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:37.961944 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.961907 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "323246b7-6b69-407e-ad2d-f5b5dd49ac1b" (UID: "323246b7-6b69-407e-ad2d-f5b5dd49ac1b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:04:37.962463 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.962438 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-data" (OuterVolumeSpecName: "istio-data") pod "323246b7-6b69-407e-ad2d-f5b5dd49ac1b" (UID: "323246b7-6b69-407e-ad2d-f5b5dd49ac1b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:37.963636 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.963613 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "323246b7-6b69-407e-ad2d-f5b5dd49ac1b" (UID: "323246b7-6b69-407e-ad2d-f5b5dd49ac1b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:04:37.963721 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.963616 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "323246b7-6b69-407e-ad2d-f5b5dd49ac1b" (UID: "323246b7-6b69-407e-ad2d-f5b5dd49ac1b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:37.963721 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.963658 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-token" (OuterVolumeSpecName: "istio-token") pod "323246b7-6b69-407e-ad2d-f5b5dd49ac1b" (UID: "323246b7-6b69-407e-ad2d-f5b5dd49ac1b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:04:37.963795 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:37.963739 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-kube-api-access-wqxfn" (OuterVolumeSpecName: "kube-api-access-wqxfn") pod "323246b7-6b69-407e-ad2d-f5b5dd49ac1b" (UID: "323246b7-6b69-407e-ad2d-f5b5dd49ac1b"). InnerVolumeSpecName "kube-api-access-wqxfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:04:38.059960 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.059934 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqxfn\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-kube-api-access-wqxfn\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:38.059960 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.059957 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:38.060125 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.059968 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:38.060125 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.059976 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:38.060125 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.059987 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:38.060125 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.059994 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:38.060125 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.060003 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:38.060125 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.060010 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:38.060125 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.060017 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/323246b7-6b69-407e-ad2d-f5b5dd49ac1b-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:38.115621 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.115588 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn"] Jul 02 22:04:38.116036 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.116022 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" containerName="istio-proxy" Jul 02 22:04:38.116113 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.116036 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" containerName="istio-proxy" Jul 02 22:04:38.116172 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.116127 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" containerName="istio-proxy" Jul 02 22:04:38.116172 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.116136 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" containerName="istio-proxy" Jul 02 22:04:38.120552 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.120525 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.124182 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.124163 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-1e99f232-openshift-default-dockercfg-tjsmw\"" Jul 02 22:04:38.131424 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.131401 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn"] Jul 02 22:04:38.261973 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.261888 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-certs\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.261973 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.261924 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-socket\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.261973 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.261950 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-podinfo\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.262188 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.262030 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-token\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.262188 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.262073 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7e2d4-03f6-4137-9810-2044c310f410-istiod-ca-cert\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.262188 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.262096 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-data\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.262188 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.262114 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-credential-socket\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.262317 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.262217 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-envoy\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.262317 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.262234 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pwd4\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-kube-api-access-4pwd4\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363049 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363014 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-envoy\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363049 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363051 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pwd4\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-kube-api-access-4pwd4\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363255 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363101 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-certs\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363255 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363128 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-socket\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363255 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363153 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-podinfo\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363255 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363188 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-token\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363255 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363217 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7e2d4-03f6-4137-9810-2044c310f410-istiod-ca-cert\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363255 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363241 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-data\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363551 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363270 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-credential-socket\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363660 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363576 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-socket\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363775 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363749 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-certs\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363877 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363852 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-data\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.363984 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.363910 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-credential-socket\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.364068 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.364051 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7e2d4-03f6-4137-9810-2044c310f410-istiod-ca-cert\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.365493 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.365473 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-envoy\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.365610 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.365593 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-podinfo\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.372110 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.372087 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pwd4\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-kube-api-access-4pwd4\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.372190 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.372163 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-token\") pod \"e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.432301 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.432277 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:38.575595 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.575570 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn"] Jul 02 22:04:38.576507 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:04:38.576481 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bc7e2d4_03f6_4137_9810_2044c310f410.slice/crio-27ced59fa052806ef189df38532981c62e850e62a4c29178d7a163b0ce3a85e1 WatchSource:0}: Error finding container 27ced59fa052806ef189df38532981c62e850e62a4c29178d7a163b0ce3a85e1: Status 404 returned error can't find the container with id 27ced59fa052806ef189df38532981c62e850e62a4c29178d7a163b0ce3a85e1 Jul 02 22:04:38.578629 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.578595 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:38.578706 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.578661 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:38.578706 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.578691 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:04:38.641448 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.641408 2547 generic.go:356] "Generic (PLEG): container finished" podID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" containerID="8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e" exitCode=0 Jul 02 22:04:38.641562 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.641466 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" Jul 02 22:04:38.641562 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.641477 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" event={"ID":"323246b7-6b69-407e-ad2d-f5b5dd49ac1b","Type":"ContainerDied","Data":"8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e"} Jul 02 22:04:38.641678 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.641560 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6" event={"ID":"323246b7-6b69-407e-ad2d-f5b5dd49ac1b","Type":"ContainerDied","Data":"cbd11c657f8bb41145288c4ebe4eaa70f5f9ebac9c0ab28dd68e836be03ab831"} Jul 02 22:04:38.641678 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.641609 2547 scope.go:117] "RemoveContainer" containerID="8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e" Jul 02 22:04:38.643138 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.643114 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" event={"ID":"6bc7e2d4-03f6-4137-9810-2044c310f410","Type":"ContainerStarted","Data":"27ced59fa052806ef189df38532981c62e850e62a4c29178d7a163b0ce3a85e1"} Jul 02 22:04:38.651427 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.651406 2547 scope.go:117] "RemoveContainer" containerID="970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355" Jul 02 22:04:38.660085 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.660056 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6"] Jul 02 22:04:38.662243 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.662215 2547 scope.go:117] "RemoveContainer" containerID="8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e" Jul 02 22:04:38.662578 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:04:38.662556 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e\": container with ID starting with 8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e not found: ID does not exist" containerID="8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e" Jul 02 22:04:38.662679 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.662588 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e"} err="failed to get container status \"8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e\": rpc error: code = NotFound desc = could not find container \"8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e\": container with ID starting with 8b34f2994dcc2c192e3a9cf192426317071e855e995dcf87d01ec7f8b9e5b45e not found: ID does not exist" Jul 02 22:04:38.662679 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.662613 2547 scope.go:117] "RemoveContainer" containerID="970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355" Jul 02 22:04:38.662873 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:04:38.662857 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355\": container with ID starting with 970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355 not found: ID does not exist" containerID="970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355" Jul 02 22:04:38.662931 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.662882 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355"} err="failed to get container status \"970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355\": rpc error: code = NotFound desc = could not find container \"970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355\": container with ID starting with 970d16028e6a6d3607cad1adc77fb63cbedd57f8c00882085cfe9143cdff3355 not found: ID does not exist" Jul 02 22:04:38.666839 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:38.666815 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0956ab05-openshift-default-86c6d6cf59-dqvk6"] Jul 02 22:04:39.651929 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:39.651897 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" event={"ID":"6bc7e2d4-03f6-4137-9810-2044c310f410","Type":"ContainerStarted","Data":"7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db"} Jul 02 22:04:39.676217 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:39.676171 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" podStartSLOduration=1.676155088 podStartE2EDuration="1.676155088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:04:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:04:39.672981059 +0000 UTC m=+2014.122819682" watchObservedRunningTime="2026-07-02 22:04:39.676155088 +0000 UTC m=+2014.125993778" Jul 02 22:04:40.010189 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:40.010111 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" path="/var/lib/kubelet/pods/323246b7-6b69-407e-ad2d-f5b5dd49ac1b/volumes" Jul 02 22:04:40.433003 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:40.432968 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:40.438246 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:40.438219 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:40.658433 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:40.658401 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:40.659241 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:40.659222 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:43.950853 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:43.950815 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn"] Jul 02 22:04:43.951322 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:43.951000 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" podUID="6bc7e2d4-03f6-4137-9810-2044c310f410" containerName="istio-proxy" containerID="cri-o://7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db" gracePeriod=30 Jul 02 22:04:49.199427 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.199296 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:49.277253 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277171 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-token\") pod \"6bc7e2d4-03f6-4137-9810-2044c310f410\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " Jul 02 22:04:49.277253 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277210 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-data\") pod \"6bc7e2d4-03f6-4137-9810-2044c310f410\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " Jul 02 22:04:49.277253 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277231 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-certs\") pod \"6bc7e2d4-03f6-4137-9810-2044c310f410\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " Jul 02 22:04:49.277560 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277260 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-socket\") pod \"6bc7e2d4-03f6-4137-9810-2044c310f410\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " Jul 02 22:04:49.277560 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277301 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4pwd4\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-kube-api-access-4pwd4\") pod \"6bc7e2d4-03f6-4137-9810-2044c310f410\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " Jul 02 22:04:49.277560 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277324 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-credential-socket\") pod \"6bc7e2d4-03f6-4137-9810-2044c310f410\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " Jul 02 22:04:49.277560 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277343 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-envoy\") pod \"6bc7e2d4-03f6-4137-9810-2044c310f410\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " Jul 02 22:04:49.277560 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277402 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-podinfo\") pod \"6bc7e2d4-03f6-4137-9810-2044c310f410\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " Jul 02 22:04:49.277560 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277503 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7e2d4-03f6-4137-9810-2044c310f410-istiod-ca-cert\") pod \"6bc7e2d4-03f6-4137-9810-2044c310f410\" (UID: \"6bc7e2d4-03f6-4137-9810-2044c310f410\") " Jul 02 22:04:49.277560 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277529 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-data" (OuterVolumeSpecName: "istio-data") pod "6bc7e2d4-03f6-4137-9810-2044c310f410" (UID: "6bc7e2d4-03f6-4137-9810-2044c310f410"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:49.277934 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277665 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6bc7e2d4-03f6-4137-9810-2044c310f410" (UID: "6bc7e2d4-03f6-4137-9810-2044c310f410"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:49.277934 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277685 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6bc7e2d4-03f6-4137-9810-2044c310f410" (UID: "6bc7e2d4-03f6-4137-9810-2044c310f410"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:49.278040 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277932 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6bc7e2d4-03f6-4137-9810-2044c310f410" (UID: "6bc7e2d4-03f6-4137-9810-2044c310f410"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:49.278040 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277941 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:49.278040 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277964 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:49.278040 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277990 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:49.278040 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.277961 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bc7e2d4-03f6-4137-9810-2044c310f410-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6bc7e2d4-03f6-4137-9810-2044c310f410" (UID: "6bc7e2d4-03f6-4137-9810-2044c310f410"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:04:49.279777 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.279750 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6bc7e2d4-03f6-4137-9810-2044c310f410" (UID: "6bc7e2d4-03f6-4137-9810-2044c310f410"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:04:49.279909 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.279847 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-kube-api-access-4pwd4" (OuterVolumeSpecName: "kube-api-access-4pwd4") pod "6bc7e2d4-03f6-4137-9810-2044c310f410" (UID: "6bc7e2d4-03f6-4137-9810-2044c310f410"). InnerVolumeSpecName "kube-api-access-4pwd4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:04:49.279965 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.279917 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6bc7e2d4-03f6-4137-9810-2044c310f410" (UID: "6bc7e2d4-03f6-4137-9810-2044c310f410"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:04:49.279965 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.279927 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-token" (OuterVolumeSpecName: "istio-token") pod "6bc7e2d4-03f6-4137-9810-2044c310f410" (UID: "6bc7e2d4-03f6-4137-9810-2044c310f410"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:04:49.378789 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.378693 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6bc7e2d4-03f6-4137-9810-2044c310f410-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:49.378789 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.378728 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:49.378789 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.378741 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:49.378789 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.378756 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pwd4\" (UniqueName: \"kubernetes.io/projected/6bc7e2d4-03f6-4137-9810-2044c310f410-kube-api-access-4pwd4\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:49.378789 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.378769 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:49.378789 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.378782 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6bc7e2d4-03f6-4137-9810-2044c310f410-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:04:49.714513 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.714477 2547 generic.go:356] "Generic (PLEG): container finished" podID="6bc7e2d4-03f6-4137-9810-2044c310f410" containerID="7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db" exitCode=0 Jul 02 22:04:49.714683 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.714522 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" Jul 02 22:04:49.714683 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.714566 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" event={"ID":"6bc7e2d4-03f6-4137-9810-2044c310f410","Type":"ContainerDied","Data":"7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db"} Jul 02 22:04:49.714683 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.714621 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn" event={"ID":"6bc7e2d4-03f6-4137-9810-2044c310f410","Type":"ContainerDied","Data":"27ced59fa052806ef189df38532981c62e850e62a4c29178d7a163b0ce3a85e1"} Jul 02 22:04:49.714683 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.714650 2547 scope.go:117] "RemoveContainer" containerID="7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db" Jul 02 22:04:49.724915 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.724896 2547 scope.go:117] "RemoveContainer" containerID="7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db" Jul 02 22:04:49.725181 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:04:49.725162 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db\": container with ID starting with 7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db not found: ID does not exist" containerID="7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db" Jul 02 22:04:49.725246 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.725194 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db"} err="failed to get container status \"7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db\": rpc error: code = NotFound desc = could not find container \"7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db\": container with ID starting with 7f1e9f3eeca264e1c674cbc74f4af335f09f64bc341320eab421522f92dd43db not found: ID does not exist" Jul 02 22:04:49.739440 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.739418 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn"] Jul 02 22:04:49.742843 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:49.742822 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-1e99f232-openshift-default-66ddf947dc-cg2vn"] Jul 02 22:04:50.010400 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:04:50.010306 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bc7e2d4-03f6-4137-9810-2044c310f410" path="/var/lib/kubelet/pods/6bc7e2d4-03f6-4137-9810-2044c310f410/volumes" Jul 02 22:06:06.107398 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:06.107361 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 22:06:06.114001 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:06.113978 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 22:06:06.115127 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:06.115095 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 22:06:06.120974 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:06.120957 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 22:06:24.819569 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.819535 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft"] Jul 02 22:06:24.819998 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.819986 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" containerName="istio-proxy" Jul 02 22:06:24.820046 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.820000 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="323246b7-6b69-407e-ad2d-f5b5dd49ac1b" containerName="istio-proxy" Jul 02 22:06:24.820046 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.820016 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bc7e2d4-03f6-4137-9810-2044c310f410" containerName="istio-proxy" Jul 02 22:06:24.820046 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.820021 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bc7e2d4-03f6-4137-9810-2044c310f410" containerName="istio-proxy" Jul 02 22:06:24.820191 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.820111 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bc7e2d4-03f6-4137-9810-2044c310f410" containerName="istio-proxy" Jul 02 22:06:24.823354 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.823339 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.825766 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.825744 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9c549bca-openshift-default-dockercfg-9gwxq\"" Jul 02 22:06:24.844196 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.844168 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft"] Jul 02 22:06:24.885717 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.885680 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-socket\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.885886 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.885736 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-data\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.885886 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.885808 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-podinfo\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.885886 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.885870 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1750bdc4-0015-450f-95e7-7d18dae9d68b-istiod-ca-cert\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.885990 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.885918 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-certs\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.885990 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.885953 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-envoy\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.885990 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.885978 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lpzd\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-kube-api-access-7lpzd\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.886090 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.886017 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-credential-socket\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.886090 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.886045 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-token\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987290 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987246 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-certs\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987517 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987300 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-envoy\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987517 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987335 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lpzd\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-kube-api-access-7lpzd\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987517 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987405 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-credential-socket\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987517 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987447 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-token\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987517 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987481 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-socket\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987807 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987521 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-data\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987807 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987549 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-podinfo\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987807 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987626 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1750bdc4-0015-450f-95e7-7d18dae9d68b-istiod-ca-cert\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987970 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987824 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-credential-socket\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987970 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987884 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-certs\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987970 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987892 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-socket\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.987970 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.987952 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-data\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.988446 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.988425 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1750bdc4-0015-450f-95e7-7d18dae9d68b-istiod-ca-cert\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.989876 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.989846 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-envoy\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:24.990093 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:24.990074 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-podinfo\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:25.000836 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:25.000802 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-token\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:25.001307 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:25.001289 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lpzd\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-kube-api-access-7lpzd\") pod \"e2e-mt-9c549bca-openshift-default-744557c7f-z5wft\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:25.133211 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:25.133116 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:25.281020 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:25.280994 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft"] Jul 02 22:06:25.281843 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:06:25.281819 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1750bdc4_0015_450f_95e7_7d18dae9d68b.slice/crio-fdac16fb1e9acea96b0eb4afd73a3392fe9fcd019c4df0dc324d646d1ff35460 WatchSource:0}: Error finding container fdac16fb1e9acea96b0eb4afd73a3392fe9fcd019c4df0dc324d646d1ff35460: Status 404 returned error can't find the container with id fdac16fb1e9acea96b0eb4afd73a3392fe9fcd019c4df0dc324d646d1ff35460 Jul 02 22:06:25.284117 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:25.284079 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:06:25.284247 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:25.284153 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:06:25.284247 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:25.284187 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:06:25.289823 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:25.289802 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" event={"ID":"1750bdc4-0015-450f-95e7-7d18dae9d68b","Type":"ContainerStarted","Data":"fdac16fb1e9acea96b0eb4afd73a3392fe9fcd019c4df0dc324d646d1ff35460"} Jul 02 22:06:26.296452 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:26.296415 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" event={"ID":"1750bdc4-0015-450f-95e7-7d18dae9d68b","Type":"ContainerStarted","Data":"397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c"} Jul 02 22:06:26.318280 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:26.318232 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" podStartSLOduration=2.318218276 podStartE2EDuration="2.318218276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:06:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:06:26.31449925 +0000 UTC m=+2120.764337900" watchObservedRunningTime="2026-07-02 22:06:26.318218276 +0000 UTC m=+2120.768056900" Jul 02 22:06:27.134226 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:27.134188 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:27.139435 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:27.139411 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:27.302389 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:27.302343 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:27.303522 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:27.303501 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:42.303886 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:42.303853 2547 patch_prober.go:28] interesting pod/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 02 22:06:42.304290 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:42.303921 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 02 22:06:42.406530 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:42.406450 2547 generic.go:356] "Generic (PLEG): container finished" podID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerID="397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c" exitCode=0 Jul 02 22:06:42.406675 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:42.406526 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" event={"ID":"1750bdc4-0015-450f-95e7-7d18dae9d68b","Type":"ContainerDied","Data":"397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c"} Jul 02 22:06:42.407043 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:42.407026 2547 scope.go:117] "RemoveContainer" containerID="397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c" Jul 02 22:06:42.408171 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:42.408142 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:06:42.408233 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:42.408213 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:06:42.408274 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:42.408243 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:06:43.414562 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:43.414527 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" event={"ID":"1750bdc4-0015-450f-95e7-7d18dae9d68b","Type":"ContainerStarted","Data":"7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e"} Jul 02 22:06:44.134030 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:44.133999 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:44.134209 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:44.134182 2547 patch_prober.go:28] interesting pod/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 02 22:06:44.134265 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:44.134232 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 02 22:06:45.134544 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:45.134502 2547 patch_prober.go:28] interesting pod/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 02 22:06:45.134962 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:45.134567 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 02 22:06:46.133995 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:46.133959 2547 patch_prober.go:28] interesting pod/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Jul 02 22:06:46.134201 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:46.134019 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Jul 02 22:06:46.751469 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:46.751434 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft"] Jul 02 22:06:46.751938 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:46.751686 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" containerID="cri-o://7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e" gracePeriod=30 Jul 02 22:06:53.014496 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.014436 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:53.070833 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.070804 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-token\") pod \"1750bdc4-0015-450f-95e7-7d18dae9d68b\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " Jul 02 22:06:53.070833 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.070846 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lpzd\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-kube-api-access-7lpzd\") pod \"1750bdc4-0015-450f-95e7-7d18dae9d68b\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " Jul 02 22:06:53.071068 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.070878 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-podinfo\") pod \"1750bdc4-0015-450f-95e7-7d18dae9d68b\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " Jul 02 22:06:53.071068 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.070903 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-envoy\") pod \"1750bdc4-0015-450f-95e7-7d18dae9d68b\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " Jul 02 22:06:53.071068 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.070928 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-certs\") pod \"1750bdc4-0015-450f-95e7-7d18dae9d68b\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " Jul 02 22:06:53.071068 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.070976 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-credential-socket\") pod \"1750bdc4-0015-450f-95e7-7d18dae9d68b\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " Jul 02 22:06:53.071068 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.071004 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-data\") pod \"1750bdc4-0015-450f-95e7-7d18dae9d68b\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " Jul 02 22:06:53.071068 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.071050 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-socket\") pod \"1750bdc4-0015-450f-95e7-7d18dae9d68b\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " Jul 02 22:06:53.071655 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.071082 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1750bdc4-0015-450f-95e7-7d18dae9d68b-istiod-ca-cert\") pod \"1750bdc4-0015-450f-95e7-7d18dae9d68b\" (UID: \"1750bdc4-0015-450f-95e7-7d18dae9d68b\") " Jul 02 22:06:53.071655 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.071366 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1750bdc4-0015-450f-95e7-7d18dae9d68b" (UID: "1750bdc4-0015-450f-95e7-7d18dae9d68b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:53.071875 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.071834 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1750bdc4-0015-450f-95e7-7d18dae9d68b" (UID: "1750bdc4-0015-450f-95e7-7d18dae9d68b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:53.072707 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.072252 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1750bdc4-0015-450f-95e7-7d18dae9d68b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1750bdc4-0015-450f-95e7-7d18dae9d68b" (UID: "1750bdc4-0015-450f-95e7-7d18dae9d68b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:06:53.072707 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.072625 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:06:53.072707 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.072647 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1750bdc4-0015-450f-95e7-7d18dae9d68b-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:06:53.072707 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.072661 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:06:53.072960 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.072748 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-data" (OuterVolumeSpecName: "istio-data") pod "1750bdc4-0015-450f-95e7-7d18dae9d68b" (UID: "1750bdc4-0015-450f-95e7-7d18dae9d68b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:53.072960 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.072804 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1750bdc4-0015-450f-95e7-7d18dae9d68b" (UID: "1750bdc4-0015-450f-95e7-7d18dae9d68b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:53.073597 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.073573 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1750bdc4-0015-450f-95e7-7d18dae9d68b" (UID: "1750bdc4-0015-450f-95e7-7d18dae9d68b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:06:53.073674 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.073641 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1750bdc4-0015-450f-95e7-7d18dae9d68b" (UID: "1750bdc4-0015-450f-95e7-7d18dae9d68b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:06:53.073674 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.073654 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-kube-api-access-7lpzd" (OuterVolumeSpecName: "kube-api-access-7lpzd") pod "1750bdc4-0015-450f-95e7-7d18dae9d68b" (UID: "1750bdc4-0015-450f-95e7-7d18dae9d68b"). InnerVolumeSpecName "kube-api-access-7lpzd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:06:53.074313 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.074293 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-token" (OuterVolumeSpecName: "istio-token") pod "1750bdc4-0015-450f-95e7-7d18dae9d68b" (UID: "1750bdc4-0015-450f-95e7-7d18dae9d68b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:06:53.173850 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.173821 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:06:53.173850 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.173847 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:06:53.173850 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.173857 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:06:53.174082 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.173865 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lpzd\" (UniqueName: \"kubernetes.io/projected/1750bdc4-0015-450f-95e7-7d18dae9d68b-kube-api-access-7lpzd\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:06:53.174082 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.173875 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:06:53.174082 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.173883 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1750bdc4-0015-450f-95e7-7d18dae9d68b-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:06:53.474129 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.474092 2547 generic.go:356] "Generic (PLEG): container finished" podID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerID="7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e" exitCode=0 Jul 02 22:06:53.474366 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.474140 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" Jul 02 22:06:53.474366 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.474170 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" event={"ID":"1750bdc4-0015-450f-95e7-7d18dae9d68b","Type":"ContainerDied","Data":"7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e"} Jul 02 22:06:53.474366 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.474230 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft" event={"ID":"1750bdc4-0015-450f-95e7-7d18dae9d68b","Type":"ContainerDied","Data":"fdac16fb1e9acea96b0eb4afd73a3392fe9fcd019c4df0dc324d646d1ff35460"} Jul 02 22:06:53.474366 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.474256 2547 scope.go:117] "RemoveContainer" containerID="7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e" Jul 02 22:06:53.484457 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.484436 2547 scope.go:117] "RemoveContainer" containerID="397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c" Jul 02 22:06:53.493795 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.493776 2547 scope.go:117] "RemoveContainer" containerID="7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e" Jul 02 22:06:53.494029 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:06:53.494010 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e\": container with ID starting with 7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e not found: ID does not exist" containerID="7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e" Jul 02 22:06:53.494076 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.494038 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e"} err="failed to get container status \"7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e\": rpc error: code = NotFound desc = could not find container \"7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e\": container with ID starting with 7366d8bb2caea8b3f6c57814c686b9551ff7b722e104e266ccbdb28d364a496e not found: ID does not exist" Jul 02 22:06:53.494076 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.494055 2547 scope.go:117] "RemoveContainer" containerID="397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c" Jul 02 22:06:53.494294 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:06:53.494279 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c\": container with ID starting with 397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c not found: ID does not exist" containerID="397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c" Jul 02 22:06:53.494338 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.494296 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c"} err="failed to get container status \"397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c\": rpc error: code = NotFound desc = could not find container \"397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c\": container with ID starting with 397ab283a34640eced3b6c4a1c4dc711b7c2a8a04342b29760c6eb2fdf35f45c not found: ID does not exist" Jul 02 22:06:53.498649 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.498623 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft"] Jul 02 22:06:53.504803 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:53.504784 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9c549bca-openshift-default-744557c7f-z5wft"] Jul 02 22:06:54.009316 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:06:54.009284 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" path="/var/lib/kubelet/pods/1750bdc4-0015-450f-95e7-7d18dae9d68b/volumes" Jul 02 22:07:06.108444 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.108410 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4"] Jul 02 22:07:06.108869 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.108856 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" Jul 02 22:07:06.108869 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.108870 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" Jul 02 22:07:06.108949 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.108880 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" Jul 02 22:07:06.108949 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.108886 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" Jul 02 22:07:06.108949 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.108946 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" Jul 02 22:07:06.113598 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.113577 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.116273 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.116256 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7bd61c91-openshift-default-dockercfg-d5lfm\"" Jul 02 22:07:06.122673 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.122649 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4"] Jul 02 22:07:06.196075 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.196028 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-certs\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.196302 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.196081 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th685\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-kube-api-access-th685\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.196302 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.196126 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-podinfo\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.196302 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.196154 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-envoy\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.196302 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.196188 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-credential-socket\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.196302 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.196244 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-socket\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.196552 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.196320 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istiod-ca-cert\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.196552 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.196413 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-token\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.196552 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.196492 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-data\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.297897 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.297857 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-credential-socket\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298097 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.297911 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-socket\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298097 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.297940 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istiod-ca-cert\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298097 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.297970 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-token\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298097 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298019 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-data\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298097 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298064 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-certs\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298097 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298085 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th685\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-kube-api-access-th685\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298465 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298104 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-podinfo\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298465 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298120 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-envoy\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298465 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298277 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-credential-socket\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298465 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298320 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-socket\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298465 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298372 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-certs\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298683 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298651 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istiod-ca-cert\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.298683 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.298673 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-data\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.300571 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.300547 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-envoy\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.300680 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.300660 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-podinfo\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.305952 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.305931 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-token\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.306040 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.306018 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th685\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-kube-api-access-th685\") pod \"e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.426022 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.425913 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:06.577704 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.577676 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4"] Jul 02 22:07:06.579069 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:07:06.579045 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86c42bb8_f5ee_4d18_a2e4_d6f9d989648c.slice/crio-0e3ab7f3d4a14813eb138fc70a9942c0dd64fd238ba12a4beddd8736e00d789b WatchSource:0}: Error finding container 0e3ab7f3d4a14813eb138fc70a9942c0dd64fd238ba12a4beddd8736e00d789b: Status 404 returned error can't find the container with id 0e3ab7f3d4a14813eb138fc70a9942c0dd64fd238ba12a4beddd8736e00d789b Jul 02 22:07:06.581681 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.581647 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:07:06.581751 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.581718 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:07:06.581795 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:06.581747 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:07:07.557342 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:07.557306 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" event={"ID":"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c","Type":"ContainerStarted","Data":"1ba38020d8afd49e980a6362121768b1865a04ae1c72d125d72a2494963e2077"} Jul 02 22:07:07.557742 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:07.557363 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" event={"ID":"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c","Type":"ContainerStarted","Data":"0e3ab7f3d4a14813eb138fc70a9942c0dd64fd238ba12a4beddd8736e00d789b"} Jul 02 22:07:07.579046 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:07.578993 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" podStartSLOduration=1.578973478 podStartE2EDuration="1.578973478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:07:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:07:07.578022533 +0000 UTC m=+2162.027861158" watchObservedRunningTime="2026-07-02 22:07:07.578973478 +0000 UTC m=+2162.028812103" Jul 02 22:07:08.427137 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:08.427098 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:08.432130 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:08.432101 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:08.564038 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:08.564001 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:08.564999 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:08.564978 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:23.565727 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:23.565690 2547 patch_prober.go:28] interesting pod/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Jul 02 22:07:23.566228 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:23.565769 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Jul 02 22:07:23.662265 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:23.662231 2547 generic.go:356] "Generic (PLEG): container finished" podID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerID="1ba38020d8afd49e980a6362121768b1865a04ae1c72d125d72a2494963e2077" exitCode=0 Jul 02 22:07:23.662448 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:23.662303 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" event={"ID":"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c","Type":"ContainerDied","Data":"1ba38020d8afd49e980a6362121768b1865a04ae1c72d125d72a2494963e2077"} Jul 02 22:07:23.662748 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:23.662734 2547 scope.go:117] "RemoveContainer" containerID="1ba38020d8afd49e980a6362121768b1865a04ae1c72d125d72a2494963e2077" Jul 02 22:07:23.663765 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:23.663738 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:07:23.663818 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:23.663809 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:07:23.663861 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:23.663842 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:07:24.675972 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:24.674949 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" event={"ID":"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c","Type":"ContainerStarted","Data":"139d604a53b71130d5846b192c67b8007abc476c653ffbdf43eb68576f6ae51a"} Jul 02 22:07:25.426348 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:25.426302 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:25.426601 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:25.426580 2547 patch_prober.go:28] interesting pod/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Jul 02 22:07:25.426649 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:25.426629 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Jul 02 22:07:26.427144 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:26.427107 2547 patch_prober.go:28] interesting pod/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Jul 02 22:07:26.427581 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:26.427166 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Jul 02 22:07:27.426911 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:27.426875 2547 patch_prober.go:28] interesting pod/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Jul 02 22:07:27.427101 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:27.426933 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Jul 02 22:07:28.440327 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:28.440294 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:28.440721 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:28.440590 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:07:28.441316 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:07:28.441297 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:09:00.042215 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:00.042132 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4"] Jul 02 22:09:00.042765 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:00.042410 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" containerID="cri-o://139d604a53b71130d5846b192c67b8007abc476c653ffbdf43eb68576f6ae51a" gracePeriod=30 Jul 02 22:09:05.301504 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.301477 2547 generic.go:356] "Generic (PLEG): container finished" podID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerID="139d604a53b71130d5846b192c67b8007abc476c653ffbdf43eb68576f6ae51a" exitCode=0 Jul 02 22:09:05.301853 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.301523 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" event={"ID":"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c","Type":"ContainerDied","Data":"139d604a53b71130d5846b192c67b8007abc476c653ffbdf43eb68576f6ae51a"} Jul 02 22:09:05.301853 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.301570 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" event={"ID":"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c","Type":"ContainerDied","Data":"0e3ab7f3d4a14813eb138fc70a9942c0dd64fd238ba12a4beddd8736e00d789b"} Jul 02 22:09:05.301853 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.301590 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e3ab7f3d4a14813eb138fc70a9942c0dd64fd238ba12a4beddd8736e00d789b" Jul 02 22:09:05.301853 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.301603 2547 scope.go:117] "RemoveContainer" containerID="1ba38020d8afd49e980a6362121768b1865a04ae1c72d125d72a2494963e2077" Jul 02 22:09:05.302892 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.302870 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:09:05.377665 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.377632 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-socket\") pod \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " Jul 02 22:09:05.377665 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.377672 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-certs\") pod \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " Jul 02 22:09:05.377903 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.377787 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-credential-socket\") pod \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " Jul 02 22:09:05.377903 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.377813 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-data\") pod \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " Jul 02 22:09:05.377903 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.377839 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istiod-ca-cert\") pod \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " Jul 02 22:09:05.377903 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.377887 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-token\") pod \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " Jul 02 22:09:05.377903 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.377894 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" (UID: "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:05.378174 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.377917 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-th685\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-kube-api-access-th685\") pod \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " Jul 02 22:09:05.378174 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.377945 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-podinfo\") pod \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " Jul 02 22:09:05.378174 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.378020 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" (UID: "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:05.378174 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.378041 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-envoy\") pod \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\" (UID: \"86c42bb8-f5ee-4d18-a2e4-d6f9d989648c\") " Jul 02 22:09:05.378394 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.378225 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" (UID: "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:09:05.378454 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.378444 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:09:05.378510 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.378462 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:09:05.378510 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.378475 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:09:05.378820 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.378795 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" (UID: "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:05.379133 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.378811 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-data" (OuterVolumeSpecName: "istio-data") pod "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" (UID: "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:05.380410 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.380359 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-kube-api-access-th685" (OuterVolumeSpecName: "kube-api-access-th685") pod "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" (UID: "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c"). InnerVolumeSpecName "kube-api-access-th685". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:05.380549 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.380529 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" (UID: "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:09:05.380802 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.380774 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" (UID: "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:09:05.380870 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.380803 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-token" (OuterVolumeSpecName: "istio-token") pod "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" (UID: "86c42bb8-f5ee-4d18-a2e4-d6f9d989648c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:09:05.479546 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.479513 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:09:05.479546 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.479540 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:09:05.479546 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.479548 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:09:05.479779 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.479556 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-th685\" (UniqueName: \"kubernetes.io/projected/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-kube-api-access-th685\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:09:05.479779 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.479567 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:09:05.479779 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:05.479576 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:09:06.312534 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:06.312502 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4" Jul 02 22:09:06.347301 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:06.347266 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4"] Jul 02 22:09:06.351134 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:06.351105 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7bd61c91-openshift-default-6b9fccd9c5-zd2w4"] Jul 02 22:09:08.009683 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:08.009651 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" path="/var/lib/kubelet/pods/86c42bb8-f5ee-4d18-a2e4-d6f9d989648c/volumes" Jul 02 22:09:54.899229 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.899197 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9"] Jul 02 22:09:54.899892 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.899873 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" Jul 02 22:09:54.899979 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.899896 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" Jul 02 22:09:54.899979 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.899910 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" Jul 02 22:09:54.899979 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.899919 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" Jul 02 22:09:54.900133 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.900037 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="1750bdc4-0015-450f-95e7-7d18dae9d68b" containerName="istio-proxy" Jul 02 22:09:54.900133 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.900050 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" Jul 02 22:09:54.900133 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.900060 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="86c42bb8-f5ee-4d18-a2e4-d6f9d989648c" containerName="istio-proxy" Jul 02 22:09:54.903322 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.903302 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:54.905907 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.905887 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f4f68d-openshift-default-dockercfg-k56j6\"" Jul 02 22:09:54.916336 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:54.916314 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9"] Jul 02 22:09:55.052141 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.052106 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-certs\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.052356 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.052153 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-istio-token\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.052356 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.052173 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-data\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.052356 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.052221 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-envoy\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.052356 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.052263 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e901e760-b2f9-4451-85da-1f491c854333-istio-podinfo\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.052356 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.052309 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-socket\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.052652 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.052413 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e901e760-b2f9-4451-85da-1f491c854333-istiod-ca-cert\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.052652 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.052458 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx84f\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-kube-api-access-kx84f\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.052652 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.052514 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-credential-socket\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.153703 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.153611 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx84f\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-kube-api-access-kx84f\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.153703 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.153688 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-credential-socket\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.153953 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.153723 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-certs\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.153953 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.153784 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-istio-token\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.153953 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.153812 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-data\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.153953 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.153848 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-envoy\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.153953 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.153877 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e901e760-b2f9-4451-85da-1f491c854333-istio-podinfo\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.153953 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.153920 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-socket\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.154296 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.153981 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e901e760-b2f9-4451-85da-1f491c854333-istiod-ca-cert\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.154296 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.154160 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-credential-socket\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.154296 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.154196 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-certs\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.154505 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.154462 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-socket\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.154505 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.154493 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-data\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.154782 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.154766 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e901e760-b2f9-4451-85da-1f491c854333-istiod-ca-cert\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.156313 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.156292 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-envoy\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.156462 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.156442 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e901e760-b2f9-4451-85da-1f491c854333-istio-podinfo\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.161458 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.161436 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-istio-token\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.161598 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.161579 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx84f\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-kube-api-access-kx84f\") pod \"e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.214880 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.214836 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:55.364840 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.364812 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9"] Jul 02 22:09:55.366994 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:09:55.366965 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode901e760_b2f9_4451_85da_1f491c854333.slice/crio-63bd547e58634573a7809b72240f2b4a60fae97c2ae98bf39f0c35810823df62 WatchSource:0}: Error finding container 63bd547e58634573a7809b72240f2b4a60fae97c2ae98bf39f0c35810823df62: Status 404 returned error can't find the container with id 63bd547e58634573a7809b72240f2b4a60fae97c2ae98bf39f0c35810823df62 Jul 02 22:09:55.369635 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.369598 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:09:55.369715 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.369680 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:09:55.369760 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.369745 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:09:55.621536 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.621492 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" event={"ID":"e901e760-b2f9-4451-85da-1f491c854333","Type":"ContainerStarted","Data":"9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35"} Jul 02 22:09:55.621717 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.621596 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" event={"ID":"e901e760-b2f9-4451-85da-1f491c854333","Type":"ContainerStarted","Data":"63bd547e58634573a7809b72240f2b4a60fae97c2ae98bf39f0c35810823df62"} Jul 02 22:09:55.643527 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:55.643471 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" podStartSLOduration=1.643449331 podStartE2EDuration="1.643449331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:09:55.640198356 +0000 UTC m=+2330.090036983" watchObservedRunningTime="2026-07-02 22:09:55.643449331 +0000 UTC m=+2330.093287953" Jul 02 22:09:56.215734 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:56.215700 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:56.220646 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:56.220620 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:56.629030 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:56.628992 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:09:56.629917 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:09:56.629893 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:10:02.773360 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.773318 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp"] Jul 02 22:10:02.777679 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.777656 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:02.780510 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.780487 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-37c632-openshift-default-dockercfg-75dww\"" Jul 02 22:10:02.789464 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.789440 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp"] Jul 02 22:10:02.931890 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.931858 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-token\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:02.932064 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.931896 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-socket\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:02.932064 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.932004 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pd44\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-kube-api-access-6pd44\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:02.932064 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.932034 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-certs\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:02.932174 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.932074 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-data\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:02.932174 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.932102 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-envoy\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:02.932174 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.932138 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07b66b1e-9859-4347-9fa4-9e0b674edd67-istiod-ca-cert\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:02.932174 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.932165 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-podinfo\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:02.932300 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:02.932193 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-credential-socket\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.033828 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.033734 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-data\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.033828 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.033782 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-envoy\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.033828 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.033809 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07b66b1e-9859-4347-9fa4-9e0b674edd67-istiod-ca-cert\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034118 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.033834 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-podinfo\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034118 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.033854 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-credential-socket\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034118 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.033888 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-token\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034118 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.033907 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-socket\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034118 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.033974 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pd44\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-kube-api-access-6pd44\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034118 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.033999 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-certs\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034469 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.034212 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-data\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034610 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.034588 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-socket\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034681 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.034606 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-credential-socket\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034681 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.034636 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07b66b1e-9859-4347-9fa4-9e0b674edd67-istiod-ca-cert\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.034786 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.034715 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-certs\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.036228 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.036205 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-envoy\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.036405 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.036373 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-podinfo\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.042022 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.041998 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-token\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.042439 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.042412 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pd44\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-kube-api-access-6pd44\") pod \"e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.090255 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.090230 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:03.242963 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.242934 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp"] Jul 02 22:10:03.244455 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:10:03.244427 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b66b1e_9859_4347_9fa4_9e0b674edd67.slice/crio-eb821804ef2f4e92301815cdcba19e7e00873a5e3548902de7b06ac79508eec9 WatchSource:0}: Error finding container eb821804ef2f4e92301815cdcba19e7e00873a5e3548902de7b06ac79508eec9: Status 404 returned error can't find the container with id eb821804ef2f4e92301815cdcba19e7e00873a5e3548902de7b06ac79508eec9 Jul 02 22:10:03.246630 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.246597 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:10:03.246712 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.246662 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:10:03.246712 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.246690 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:10:03.678581 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.678540 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" event={"ID":"07b66b1e-9859-4347-9fa4-9e0b674edd67","Type":"ContainerStarted","Data":"edb550a6260d313ac70efe5bbc0c7289b753f6698d3dc15ee369fcaf6fcf05ca"} Jul 02 22:10:03.678765 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.678602 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" event={"ID":"07b66b1e-9859-4347-9fa4-9e0b674edd67","Type":"ContainerStarted","Data":"eb821804ef2f4e92301815cdcba19e7e00873a5e3548902de7b06ac79508eec9"} Jul 02 22:10:03.706210 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:03.706163 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" podStartSLOduration=1.706147409 podStartE2EDuration="1.706147409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:10:03.702912321 +0000 UTC m=+2338.152750946" watchObservedRunningTime="2026-07-02 22:10:03.706147409 +0000 UTC m=+2338.155986033" Jul 02 22:10:04.090436 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:04.090400 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:04.095284 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:04.095258 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:04.684619 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:04.684587 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:04.685682 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:04.685663 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:12.670651 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:12.670600 2547 patch_prober.go:28] interesting pod/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 02 22:10:12.671328 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:12.671286 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 02 22:10:13.751486 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:13.751362 2547 generic.go:356] "Generic (PLEG): container finished" podID="e901e760-b2f9-4451-85da-1f491c854333" containerID="9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35" exitCode=0 Jul 02 22:10:13.751486 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:13.751447 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" event={"ID":"e901e760-b2f9-4451-85da-1f491c854333","Type":"ContainerDied","Data":"9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35"} Jul 02 22:10:13.752066 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:13.751943 2547 scope.go:117] "RemoveContainer" containerID="9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35" Jul 02 22:10:13.753038 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:13.753010 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:10:13.753115 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:13.753090 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:10:13.753162 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:13.753125 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:10:14.759228 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:14.759196 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" event={"ID":"e901e760-b2f9-4451-85da-1f491c854333","Type":"ContainerStarted","Data":"4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef"} Jul 02 22:10:15.215344 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:15.215290 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:10:15.215593 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:15.215468 2547 patch_prober.go:28] interesting pod/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Jul 02 22:10:15.215593 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:15.215514 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Jul 02 22:10:16.215847 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:16.215813 2547 patch_prober.go:28] interesting pod/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Jul 02 22:10:16.216246 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:16.215865 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Jul 02 22:10:17.215952 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:17.215917 2547 patch_prober.go:28] interesting pod/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Jul 02 22:10:17.216340 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:17.215968 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Jul 02 22:10:18.215811 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:18.215766 2547 patch_prober.go:28] interesting pod/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Jul 02 22:10:18.216002 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:18.215834 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Jul 02 22:10:19.219110 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.219078 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:10:19.219532 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.219431 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:10:19.220210 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.220185 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:10:19.685979 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.685949 2547 patch_prober.go:28] interesting pod/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 02 22:10:19.686167 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.686027 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 02 22:10:19.805084 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.804989 2547 generic.go:356] "Generic (PLEG): container finished" podID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerID="edb550a6260d313ac70efe5bbc0c7289b753f6698d3dc15ee369fcaf6fcf05ca" exitCode=0 Jul 02 22:10:19.805084 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.805064 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" event={"ID":"07b66b1e-9859-4347-9fa4-9e0b674edd67","Type":"ContainerDied","Data":"edb550a6260d313ac70efe5bbc0c7289b753f6698d3dc15ee369fcaf6fcf05ca"} Jul 02 22:10:19.805590 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.805577 2547 scope.go:117] "RemoveContainer" containerID="edb550a6260d313ac70efe5bbc0c7289b753f6698d3dc15ee369fcaf6fcf05ca" Jul 02 22:10:19.806535 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.806510 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:10:19.806598 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.806572 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:10:19.806634 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:19.806616 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:10:20.812708 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:20.812675 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" event={"ID":"07b66b1e-9859-4347-9fa4-9e0b674edd67","Type":"ContainerStarted","Data":"1753ccf011a01f32f331c39c9bf20d1d9bfc78f4708da07d4e3c782fa0431b9f"} Jul 02 22:10:21.090713 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:21.090614 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:21.091071 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:21.091043 2547 patch_prober.go:28] interesting pod/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 02 22:10:21.091182 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:21.091102 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 02 22:10:22.091372 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:22.091320 2547 patch_prober.go:28] interesting pod/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 02 22:10:22.092075 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:22.092034 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 02 22:10:23.090786 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:23.090741 2547 patch_prober.go:28] interesting pod/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 02 22:10:23.091014 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:23.090811 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 02 22:10:24.090698 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:24.090652 2547 patch_prober.go:28] interesting pod/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Jul 02 22:10:24.091111 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:24.090736 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Jul 02 22:10:25.095166 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:25.095130 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:25.095707 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:25.095489 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:10:25.096166 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:10:25.096150 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:11:06.149169 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:11:06.149139 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 22:11:06.157128 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:11:06.157099 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 22:11:06.158506 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:11:06.158486 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 22:11:06.165611 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:11:06.165594 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 22:11:57.465940 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:11:57.465848 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9"] Jul 02 22:11:57.466558 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:11:57.466163 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" containerID="cri-o://4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef" gracePeriod=30 Jul 02 22:12:02.728339 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.728274 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:12:02.843688 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.843652 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-certs\") pod \"e901e760-b2f9-4451-85da-1f491c854333\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " Jul 02 22:12:02.843688 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.843689 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-envoy\") pod \"e901e760-b2f9-4451-85da-1f491c854333\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " Jul 02 22:12:02.843961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.843753 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-istio-token\") pod \"e901e760-b2f9-4451-85da-1f491c854333\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " Jul 02 22:12:02.843961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.843780 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e901e760-b2f9-4451-85da-1f491c854333-istio-podinfo\") pod \"e901e760-b2f9-4451-85da-1f491c854333\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " Jul 02 22:12:02.843961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.843832 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-socket\") pod \"e901e760-b2f9-4451-85da-1f491c854333\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " Jul 02 22:12:02.843961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.843866 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-data\") pod \"e901e760-b2f9-4451-85da-1f491c854333\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " Jul 02 22:12:02.843961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.843908 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e901e760-b2f9-4451-85da-1f491c854333-istiod-ca-cert\") pod \"e901e760-b2f9-4451-85da-1f491c854333\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " Jul 02 22:12:02.843961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.843943 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx84f\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-kube-api-access-kx84f\") pod \"e901e760-b2f9-4451-85da-1f491c854333\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " Jul 02 22:12:02.844344 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.843982 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-credential-socket\") pod \"e901e760-b2f9-4451-85da-1f491c854333\" (UID: \"e901e760-b2f9-4451-85da-1f491c854333\") " Jul 02 22:12:02.844344 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.844024 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e901e760-b2f9-4451-85da-1f491c854333" (UID: "e901e760-b2f9-4451-85da-1f491c854333"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:02.844344 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.844201 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e901e760-b2f9-4451-85da-1f491c854333" (UID: "e901e760-b2f9-4451-85da-1f491c854333"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:02.844344 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.844295 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e901e760-b2f9-4451-85da-1f491c854333-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e901e760-b2f9-4451-85da-1f491c854333" (UID: "e901e760-b2f9-4451-85da-1f491c854333"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:12:02.844614 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.844464 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e901e760-b2f9-4451-85da-1f491c854333" (UID: "e901e760-b2f9-4451-85da-1f491c854333"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:02.844614 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.844536 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:02.844614 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.844556 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:02.844614 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.844569 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e901e760-b2f9-4451-85da-1f491c854333-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:02.845169 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.845145 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-data" (OuterVolumeSpecName: "istio-data") pod "e901e760-b2f9-4451-85da-1f491c854333" (UID: "e901e760-b2f9-4451-85da-1f491c854333"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:02.846131 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.846110 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e901e760-b2f9-4451-85da-1f491c854333-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e901e760-b2f9-4451-85da-1f491c854333" (UID: "e901e760-b2f9-4451-85da-1f491c854333"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:12:02.846225 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.846173 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-kube-api-access-kx84f" (OuterVolumeSpecName: "kube-api-access-kx84f") pod "e901e760-b2f9-4451-85da-1f491c854333" (UID: "e901e760-b2f9-4451-85da-1f491c854333"). InnerVolumeSpecName "kube-api-access-kx84f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:02.846326 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.846312 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e901e760-b2f9-4451-85da-1f491c854333" (UID: "e901e760-b2f9-4451-85da-1f491c854333"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:02.846365 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.846319 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-istio-token" (OuterVolumeSpecName: "istio-token") pod "e901e760-b2f9-4451-85da-1f491c854333" (UID: "e901e760-b2f9-4451-85da-1f491c854333"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:02.945653 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.945623 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:02.945653 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.945649 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx84f\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-kube-api-access-kx84f\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:02.945653 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.945660 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:02.945885 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.945669 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e901e760-b2f9-4451-85da-1f491c854333-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:02.945885 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.945677 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e901e760-b2f9-4451-85da-1f491c854333-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:02.945885 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:02.945685 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e901e760-b2f9-4451-85da-1f491c854333-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:03.436367 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.436333 2547 generic.go:356] "Generic (PLEG): container finished" podID="e901e760-b2f9-4451-85da-1f491c854333" containerID="4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef" exitCode=0 Jul 02 22:12:03.436566 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.436409 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" Jul 02 22:12:03.436566 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.436411 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" event={"ID":"e901e760-b2f9-4451-85da-1f491c854333","Type":"ContainerDied","Data":"4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef"} Jul 02 22:12:03.436566 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.436464 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9" event={"ID":"e901e760-b2f9-4451-85da-1f491c854333","Type":"ContainerDied","Data":"63bd547e58634573a7809b72240f2b4a60fae97c2ae98bf39f0c35810823df62"} Jul 02 22:12:03.436566 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.436488 2547 scope.go:117] "RemoveContainer" containerID="4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef" Jul 02 22:12:03.453798 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.453775 2547 scope.go:117] "RemoveContainer" containerID="9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35" Jul 02 22:12:03.463397 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.463347 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9"] Jul 02 22:12:03.463879 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.463857 2547 scope.go:117] "RemoveContainer" containerID="4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef" Jul 02 22:12:03.464143 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:12:03.464122 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef\": container with ID starting with 4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef not found: ID does not exist" containerID="4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef" Jul 02 22:12:03.464224 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.464156 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef"} err="failed to get container status \"4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef\": rpc error: code = NotFound desc = could not find container \"4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef\": container with ID starting with 4f3be8eb333ef9998346b7a2d42b6c79c4c3284fad59e642be7800e1cc3350ef not found: ID does not exist" Jul 02 22:12:03.464224 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.464182 2547 scope.go:117] "RemoveContainer" containerID="9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35" Jul 02 22:12:03.464504 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:12:03.464478 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35\": container with ID starting with 9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35 not found: ID does not exist" containerID="9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35" Jul 02 22:12:03.464596 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.464509 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35"} err="failed to get container status \"9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35\": rpc error: code = NotFound desc = could not find container \"9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35\": container with ID starting with 9573300cd61a5017163d9dd035ebce3f12059a9e860be608eb693462953c0f35 not found: ID does not exist" Jul 02 22:12:03.472864 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:03.472844 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f4f68d-openshift-default-5b74f48dfc-c5gm9"] Jul 02 22:12:04.009484 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:04.009449 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e901e760-b2f9-4451-85da-1f491c854333" path="/var/lib/kubelet/pods/e901e760-b2f9-4451-85da-1f491c854333/volumes" Jul 02 22:12:04.282183 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:04.282097 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp"] Jul 02 22:12:04.282575 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:04.282541 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" containerID="cri-o://1753ccf011a01f32f331c39c9bf20d1d9bfc78f4708da07d4e3c782fa0431b9f" gracePeriod=30 Jul 02 22:12:05.170980 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.170948 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz"] Jul 02 22:12:05.171613 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.171595 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" Jul 02 22:12:05.171613 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.171615 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" Jul 02 22:12:05.171759 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.171629 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" Jul 02 22:12:05.171759 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.171636 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" Jul 02 22:12:05.171860 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.171770 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" Jul 02 22:12:05.171860 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.171784 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="e901e760-b2f9-4451-85da-1f491c854333" containerName="istio-proxy" Jul 02 22:12:05.176364 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.176343 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.179027 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.179003 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-002aff-openshift-default-dockercfg-rpwz8\"" Jul 02 22:12:05.187252 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.187230 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz"] Jul 02 22:12:05.270253 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.270212 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-token\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.270441 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.270279 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-podinfo\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.270441 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.270312 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvphf\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-kube-api-access-kvphf\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.270441 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.270367 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-certs\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.270441 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.270410 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istiod-ca-cert\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.270598 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.270468 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-data\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.270598 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.270505 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-socket\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.270598 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.270538 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-envoy\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.270598 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.270585 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-credential-socket\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372091 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372057 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-certs\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372091 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372092 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istiod-ca-cert\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372290 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372209 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-data\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372290 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372234 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-socket\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372290 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372254 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-envoy\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372290 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372282 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-credential-socket\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372504 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372331 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-token\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372504 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372473 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-podinfo\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372613 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372512 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvphf\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-kube-api-access-kvphf\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372674 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372632 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-socket\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372732 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372665 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-credential-socket\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372787 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372732 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-data\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372827 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372786 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-certs\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.372938 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.372919 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istiod-ca-cert\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.374562 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.374542 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-envoy\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.374844 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.374825 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-podinfo\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.383484 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.383451 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-token\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.383846 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.383821 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvphf\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-kube-api-access-kvphf\") pod \"e2e-api-a-002aff-openshift-default-869dccd45-jvgvz\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.487312 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.487214 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:05.639539 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.639508 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz"] Jul 02 22:12:05.640798 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:12:05.640772 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee1f990_34c6_460b_aa1c_de3950bf4e9c.slice/crio-7f54fe0b56e9c68b42bda1bfe71505a6ad48a5d2e98c31acd3613f85d6d11205 WatchSource:0}: Error finding container 7f54fe0b56e9c68b42bda1bfe71505a6ad48a5d2e98c31acd3613f85d6d11205: Status 404 returned error can't find the container with id 7f54fe0b56e9c68b42bda1bfe71505a6ad48a5d2e98c31acd3613f85d6d11205 Jul 02 22:12:05.643193 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.643113 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:05.643266 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.643196 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:05.643266 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:05.643239 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:06.459510 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:06.459472 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" event={"ID":"aee1f990-34c6-460b-aa1c-de3950bf4e9c","Type":"ContainerStarted","Data":"d987c6f17915064cb0c40a8214795450e202945cfaaf64c02c0afb575205c28a"} Jul 02 22:12:06.459927 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:06.459537 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" event={"ID":"aee1f990-34c6-460b-aa1c-de3950bf4e9c","Type":"ContainerStarted","Data":"7f54fe0b56e9c68b42bda1bfe71505a6ad48a5d2e98c31acd3613f85d6d11205"} Jul 02 22:12:06.481453 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:06.481401 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" podStartSLOduration=1.481366676 podStartE2EDuration="1.481366676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:06.479543205 +0000 UTC m=+2460.929381844" watchObservedRunningTime="2026-07-02 22:12:06.481366676 +0000 UTC m=+2460.931205365" Jul 02 22:12:06.488391 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:06.488355 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:06.493485 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:06.493465 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:07.465260 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:07.465223 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:07.466102 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:07.466085 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:09.484368 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.484172 2547 generic.go:356] "Generic (PLEG): container finished" podID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerID="1753ccf011a01f32f331c39c9bf20d1d9bfc78f4708da07d4e3c782fa0431b9f" exitCode=0 Jul 02 22:12:09.484368 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.484209 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" event={"ID":"07b66b1e-9859-4347-9fa4-9e0b674edd67","Type":"ContainerDied","Data":"1753ccf011a01f32f331c39c9bf20d1d9bfc78f4708da07d4e3c782fa0431b9f"} Jul 02 22:12:09.484368 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.484294 2547 scope.go:117] "RemoveContainer" containerID="edb550a6260d313ac70efe5bbc0c7289b753f6698d3dc15ee369fcaf6fcf05ca" Jul 02 22:12:09.558873 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.558850 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:12:09.616440 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616413 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-envoy\") pod \"07b66b1e-9859-4347-9fa4-9e0b674edd67\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " Jul 02 22:12:09.616590 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616453 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-data\") pod \"07b66b1e-9859-4347-9fa4-9e0b674edd67\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " Jul 02 22:12:09.616590 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616476 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pd44\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-kube-api-access-6pd44\") pod \"07b66b1e-9859-4347-9fa4-9e0b674edd67\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " Jul 02 22:12:09.616590 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616512 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-socket\") pod \"07b66b1e-9859-4347-9fa4-9e0b674edd67\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " Jul 02 22:12:09.616590 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616533 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-certs\") pod \"07b66b1e-9859-4347-9fa4-9e0b674edd67\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " Jul 02 22:12:09.616828 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616606 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07b66b1e-9859-4347-9fa4-9e0b674edd67-istiod-ca-cert\") pod \"07b66b1e-9859-4347-9fa4-9e0b674edd67\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " Jul 02 22:12:09.616828 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616643 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-token\") pod \"07b66b1e-9859-4347-9fa4-9e0b674edd67\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " Jul 02 22:12:09.616828 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616693 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-podinfo\") pod \"07b66b1e-9859-4347-9fa4-9e0b674edd67\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " Jul 02 22:12:09.616828 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616746 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-credential-socket\") pod \"07b66b1e-9859-4347-9fa4-9e0b674edd67\" (UID: \"07b66b1e-9859-4347-9fa4-9e0b674edd67\") " Jul 02 22:12:09.617017 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.616835 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "07b66b1e-9859-4347-9fa4-9e0b674edd67" (UID: "07b66b1e-9859-4347-9fa4-9e0b674edd67"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:09.617187 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.617162 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07b66b1e-9859-4347-9fa4-9e0b674edd67-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "07b66b1e-9859-4347-9fa4-9e0b674edd67" (UID: "07b66b1e-9859-4347-9fa4-9e0b674edd67"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:12:09.617287 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.617210 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-data" (OuterVolumeSpecName: "istio-data") pod "07b66b1e-9859-4347-9fa4-9e0b674edd67" (UID: "07b66b1e-9859-4347-9fa4-9e0b674edd67"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:09.617287 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.617237 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "07b66b1e-9859-4347-9fa4-9e0b674edd67" (UID: "07b66b1e-9859-4347-9fa4-9e0b674edd67"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:09.617436 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.617407 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.617491 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.617461 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "07b66b1e-9859-4347-9fa4-9e0b674edd67" (UID: "07b66b1e-9859-4347-9fa4-9e0b674edd67"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:09.618746 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.618720 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "07b66b1e-9859-4347-9fa4-9e0b674edd67" (UID: "07b66b1e-9859-4347-9fa4-9e0b674edd67"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:09.619059 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.619040 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "07b66b1e-9859-4347-9fa4-9e0b674edd67" (UID: "07b66b1e-9859-4347-9fa4-9e0b674edd67"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:12:09.619133 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.619112 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-token" (OuterVolumeSpecName: "istio-token") pod "07b66b1e-9859-4347-9fa4-9e0b674edd67" (UID: "07b66b1e-9859-4347-9fa4-9e0b674edd67"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:09.619491 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.619475 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-kube-api-access-6pd44" (OuterVolumeSpecName: "kube-api-access-6pd44") pod "07b66b1e-9859-4347-9fa4-9e0b674edd67" (UID: "07b66b1e-9859-4347-9fa4-9e0b674edd67"). InnerVolumeSpecName "kube-api-access-6pd44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:09.718172 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.718143 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/07b66b1e-9859-4347-9fa4-9e0b674edd67-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.718172 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.718168 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.718172 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.718178 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.718407 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.718186 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.718407 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.718194 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.718407 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.718202 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.718407 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.718209 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pd44\" (UniqueName: \"kubernetes.io/projected/07b66b1e-9859-4347-9fa4-9e0b674edd67-kube-api-access-6pd44\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:09.718407 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:09.718218 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/07b66b1e-9859-4347-9fa4-9e0b674edd67-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:10.491626 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:10.491595 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" Jul 02 22:12:10.492071 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:10.491630 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp" event={"ID":"07b66b1e-9859-4347-9fa4-9e0b674edd67","Type":"ContainerDied","Data":"eb821804ef2f4e92301815cdcba19e7e00873a5e3548902de7b06ac79508eec9"} Jul 02 22:12:10.492071 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:10.491687 2547 scope.go:117] "RemoveContainer" containerID="1753ccf011a01f32f331c39c9bf20d1d9bfc78f4708da07d4e3c782fa0431b9f" Jul 02 22:12:10.511106 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:10.511080 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp"] Jul 02 22:12:10.517569 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:10.517542 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-37c632-openshift-default-5d69777d75-lldjp"] Jul 02 22:12:12.009463 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:12.009431 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" path="/var/lib/kubelet/pods/07b66b1e-9859-4347-9fa4-9e0b674edd67/volumes" Jul 02 22:12:13.730116 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.730073 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b"] Jul 02 22:12:13.730752 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.730732 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" Jul 02 22:12:13.730874 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.730754 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" Jul 02 22:12:13.730874 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.730795 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" Jul 02 22:12:13.730874 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.730804 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" Jul 02 22:12:13.731040 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.730922 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" Jul 02 22:12:13.731040 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.730934 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="07b66b1e-9859-4347-9fa4-9e0b674edd67" containerName="istio-proxy" Jul 02 22:12:13.734329 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.734307 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.736842 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.736824 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-6f0ab8-openshift-default-dockercfg-v6tkk\"" Jul 02 22:12:13.747799 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.747775 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b"] Jul 02 22:12:13.854283 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.854248 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-certs\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.854467 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.854300 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-socket\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.854467 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.854371 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhcn4\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-kube-api-access-mhcn4\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.854467 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.854426 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istiod-ca-cert\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.854578 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.854480 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-data\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.854578 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.854520 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-credential-socket\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.854578 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.854572 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-podinfo\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.854671 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.854595 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-token\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.854671 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.854614 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-envoy\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.955637 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.955603 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhcn4\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-kube-api-access-mhcn4\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.955637 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.955642 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istiod-ca-cert\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.955866 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.955674 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-data\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.955866 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.955728 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-credential-socket\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.955866 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.955780 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-podinfo\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.955866 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.955796 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-token\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.955866 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.955815 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-envoy\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.956139 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.955870 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-certs\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.956139 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.955924 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-socket\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.956298 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.956275 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-socket\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.956452 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.956311 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-certs\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.956452 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.956360 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-credential-socket\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.956543 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.956492 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-data\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.956543 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.956526 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istiod-ca-cert\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.958363 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.958332 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-envoy\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.958523 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.958503 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-podinfo\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.964446 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.964425 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-token\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:13.964632 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:13.964612 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhcn4\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-kube-api-access-mhcn4\") pod \"e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:14.046742 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:14.046717 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:14.194342 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:12:14.194311 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c64fe6e_b081_44fe_9f80_1e6e36d265a9.slice/crio-4679e81680ef983934b1c111e29a8b1040775bc3a0ffdf95babf8a66a68062ca WatchSource:0}: Error finding container 4679e81680ef983934b1c111e29a8b1040775bc3a0ffdf95babf8a66a68062ca: Status 404 returned error can't find the container with id 4679e81680ef983934b1c111e29a8b1040775bc3a0ffdf95babf8a66a68062ca Jul 02 22:12:14.194738 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:14.194709 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b"] Jul 02 22:12:14.196698 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:14.196670 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:14.196784 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:14.196724 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:14.196784 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:14.196752 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:14.520058 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:14.519965 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" event={"ID":"9c64fe6e-b081-44fe-9f80-1e6e36d265a9","Type":"ContainerStarted","Data":"94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c"} Jul 02 22:12:14.520058 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:14.520032 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" event={"ID":"9c64fe6e-b081-44fe-9f80-1e6e36d265a9","Type":"ContainerStarted","Data":"4679e81680ef983934b1c111e29a8b1040775bc3a0ffdf95babf8a66a68062ca"} Jul 02 22:12:14.538881 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:14.538835 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" podStartSLOduration=1.538818412 podStartE2EDuration="1.538818412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:14.537492024 +0000 UTC m=+2468.987330648" watchObservedRunningTime="2026-07-02 22:12:14.538818412 +0000 UTC m=+2468.988657037" Jul 02 22:12:15.047373 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:15.047339 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:15.052517 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:15.052493 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:15.527196 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:15.527107 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:15.528150 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:15.528128 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:22.806890 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:22.805622 2547 patch_prober.go:28] interesting pod/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": read tcp 10.132.0.2:41358->10.132.0.75:15021: read: connection reset by peer" start-of-body= Jul 02 22:12:22.806890 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:22.805688 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": read tcp 10.132.0.2:41358->10.132.0.75:15021: read: connection reset by peer" Jul 02 22:12:23.589594 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:23.589525 2547 generic.go:356] "Generic (PLEG): container finished" podID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerID="d987c6f17915064cb0c40a8214795450e202945cfaaf64c02c0afb575205c28a" exitCode=0 Jul 02 22:12:23.589594 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:23.589583 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" event={"ID":"aee1f990-34c6-460b-aa1c-de3950bf4e9c","Type":"ContainerDied","Data":"d987c6f17915064cb0c40a8214795450e202945cfaaf64c02c0afb575205c28a"} Jul 02 22:12:23.590082 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:23.590065 2547 scope.go:117] "RemoveContainer" containerID="d987c6f17915064cb0c40a8214795450e202945cfaaf64c02c0afb575205c28a" Jul 02 22:12:23.591161 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:23.591131 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:23.591248 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:23.591206 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:23.591248 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:23.591242 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:24.598964 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:24.598930 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" event={"ID":"aee1f990-34c6-460b-aa1c-de3950bf4e9c","Type":"ContainerStarted","Data":"deb546cfbf49cb4344c245a8dfc8d7c13c22ce94186e6e842c92fcf76377a260"} Jul 02 22:12:25.493233 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:25.493195 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:25.493472 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:25.493314 2547 patch_prober.go:28] interesting pod/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 02 22:12:25.493472 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:25.493354 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 02 22:12:26.488572 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:26.488523 2547 patch_prober.go:28] interesting pod/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 02 22:12:26.489090 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:26.488596 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 02 22:12:27.488437 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:27.488390 2547 patch_prober.go:28] interesting pod/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 02 22:12:27.488690 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:27.488469 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 02 22:12:28.488330 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:28.487816 2547 patch_prober.go:28] interesting pod/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Jul 02 22:12:28.488330 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:28.487886 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Jul 02 22:12:28.634042 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:28.634012 2547 generic.go:356] "Generic (PLEG): container finished" podID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerID="94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c" exitCode=0 Jul 02 22:12:28.634497 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:28.634091 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" event={"ID":"9c64fe6e-b081-44fe-9f80-1e6e36d265a9","Type":"ContainerDied","Data":"94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c"} Jul 02 22:12:28.634585 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:28.634557 2547 scope.go:117] "RemoveContainer" containerID="94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c" Jul 02 22:12:28.635585 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:28.635560 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:28.635663 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:28.635629 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:28.635705 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:28.635662 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:29.493095 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:29.493057 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:29.493462 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:29.493437 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:29.494415 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:29.494364 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:29.641903 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:29.641869 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" event={"ID":"9c64fe6e-b081-44fe-9f80-1e6e36d265a9","Type":"ContainerStarted","Data":"95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33"} Jul 02 22:12:30.047173 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:30.047142 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:30.048174 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:30.048144 2547 patch_prober.go:28] interesting pod/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Jul 02 22:12:30.048319 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:30.048216 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Jul 02 22:12:30.527550 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:30.527457 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:31.047718 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:31.047671 2547 patch_prober.go:28] interesting pod/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Jul 02 22:12:31.048200 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:31.047739 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Jul 02 22:12:32.047003 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:32.046970 2547 patch_prober.go:28] interesting pod/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Jul 02 22:12:32.047201 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:32.047021 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Jul 02 22:12:33.047265 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:33.047228 2547 patch_prober.go:28] interesting pod/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Jul 02 22:12:33.047726 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:33.047283 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Jul 02 22:12:34.051320 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:34.051291 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:34.052643 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:34.052620 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:35.494316 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:35.494280 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz"] Jul 02 22:12:35.495169 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:35.495135 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" containerID="cri-o://deb546cfbf49cb4344c245a8dfc8d7c13c22ce94186e6e842c92fcf76377a260" gracePeriod=30 Jul 02 22:12:40.712570 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.712536 2547 generic.go:356] "Generic (PLEG): container finished" podID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerID="deb546cfbf49cb4344c245a8dfc8d7c13c22ce94186e6e842c92fcf76377a260" exitCode=0 Jul 02 22:12:40.712960 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.712617 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" event={"ID":"aee1f990-34c6-460b-aa1c-de3950bf4e9c","Type":"ContainerDied","Data":"deb546cfbf49cb4344c245a8dfc8d7c13c22ce94186e6e842c92fcf76377a260"} Jul 02 22:12:40.712960 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.712684 2547 scope.go:117] "RemoveContainer" containerID="d987c6f17915064cb0c40a8214795450e202945cfaaf64c02c0afb575205c28a" Jul 02 22:12:40.769366 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.769300 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:40.853664 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.853627 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istiod-ca-cert\") pod \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " Jul 02 22:12:40.853664 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.853667 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvphf\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-kube-api-access-kvphf\") pod \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " Jul 02 22:12:40.853876 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.853724 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-token\") pod \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " Jul 02 22:12:40.853876 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.853839 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-data\") pod \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " Jul 02 22:12:40.853876 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.853873 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-certs\") pod \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " Jul 02 22:12:40.853990 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.853895 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-socket\") pod \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " Jul 02 22:12:40.853990 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.853935 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-envoy\") pod \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " Jul 02 22:12:40.853990 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.853966 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-podinfo\") pod \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " Jul 02 22:12:40.854140 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854009 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-credential-socket\") pod \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\" (UID: \"aee1f990-34c6-460b-aa1c-de3950bf4e9c\") " Jul 02 22:12:40.854140 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854102 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aee1f990-34c6-460b-aa1c-de3950bf4e9c" (UID: "aee1f990-34c6-460b-aa1c-de3950bf4e9c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:40.854254 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854139 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aee1f990-34c6-460b-aa1c-de3950bf4e9c" (UID: "aee1f990-34c6-460b-aa1c-de3950bf4e9c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:40.854254 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854148 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aee1f990-34c6-460b-aa1c-de3950bf4e9c" (UID: "aee1f990-34c6-460b-aa1c-de3950bf4e9c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:12:40.854367 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854324 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aee1f990-34c6-460b-aa1c-de3950bf4e9c" (UID: "aee1f990-34c6-460b-aa1c-de3950bf4e9c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:40.854481 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854463 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:40.854547 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854484 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:40.854547 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854497 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:40.854547 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854509 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:40.854790 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.854764 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-data" (OuterVolumeSpecName: "istio-data") pod "aee1f990-34c6-460b-aa1c-de3950bf4e9c" (UID: "aee1f990-34c6-460b-aa1c-de3950bf4e9c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:40.856516 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.856463 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aee1f990-34c6-460b-aa1c-de3950bf4e9c" (UID: "aee1f990-34c6-460b-aa1c-de3950bf4e9c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:40.856516 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.856471 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-token" (OuterVolumeSpecName: "istio-token") pod "aee1f990-34c6-460b-aa1c-de3950bf4e9c" (UID: "aee1f990-34c6-460b-aa1c-de3950bf4e9c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:40.856516 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.856491 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-kube-api-access-kvphf" (OuterVolumeSpecName: "kube-api-access-kvphf") pod "aee1f990-34c6-460b-aa1c-de3950bf4e9c" (UID: "aee1f990-34c6-460b-aa1c-de3950bf4e9c"). InnerVolumeSpecName "kube-api-access-kvphf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:40.856744 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.856599 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aee1f990-34c6-460b-aa1c-de3950bf4e9c" (UID: "aee1f990-34c6-460b-aa1c-de3950bf4e9c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:12:40.955766 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.955737 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:40.955766 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.955763 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:40.955766 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.955772 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:40.955987 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.955780 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvphf\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-kube-api-access-kvphf\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:40.955987 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:40.955790 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee1f990-34c6-460b-aa1c-de3950bf4e9c-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:41.720327 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:41.720299 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" Jul 02 22:12:41.720797 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:41.720349 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz" event={"ID":"aee1f990-34c6-460b-aa1c-de3950bf4e9c","Type":"ContainerDied","Data":"7f54fe0b56e9c68b42bda1bfe71505a6ad48a5d2e98c31acd3613f85d6d11205"} Jul 02 22:12:41.720797 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:41.720421 2547 scope.go:117] "RemoveContainer" containerID="deb546cfbf49cb4344c245a8dfc8d7c13c22ce94186e6e842c92fcf76377a260" Jul 02 22:12:41.742420 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:41.742393 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz"] Jul 02 22:12:41.754463 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:41.754435 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-002aff-openshift-default-869dccd45-jvgvz"] Jul 02 22:12:42.009847 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:42.009751 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" path="/var/lib/kubelet/pods/aee1f990-34c6-460b-aa1c-de3950bf4e9c/volumes" Jul 02 22:12:42.480994 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:42.480576 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b"] Jul 02 22:12:42.480994 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:42.480802 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" containerID="cri-o://95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33" gracePeriod=30 Jul 02 22:12:43.340764 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.340729 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5"] Jul 02 22:12:43.341215 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.341201 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" Jul 02 22:12:43.341266 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.341217 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" Jul 02 22:12:43.341266 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.341242 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" Jul 02 22:12:43.341266 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.341247 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" Jul 02 22:12:43.341365 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.341320 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" Jul 02 22:12:43.344472 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.344457 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.347125 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.347105 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-77374e-openshift-default-dockercfg-cxtc6\"" Jul 02 22:12:43.355785 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.355757 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5"] Jul 02 22:12:43.482513 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.482469 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istiod-ca-cert\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.482715 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.482569 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-token\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.482715 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.482621 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-data\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.482715 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.482689 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-socket\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.482857 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.482727 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-certs\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.482857 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.482776 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-credential-socket\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.482857 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.482817 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-envoy\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.482857 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.482844 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-podinfo\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.483024 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.482923 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn9tg\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-kube-api-access-fn9tg\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584192 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584155 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-token\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584192 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584202 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-data\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584481 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584235 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-socket\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584481 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584268 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-certs\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584481 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584287 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-credential-socket\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584481 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584312 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-envoy\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584481 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584342 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-podinfo\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584481 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584420 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn9tg\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-kube-api-access-fn9tg\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584481 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584469 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istiod-ca-cert\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584831 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584676 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-data\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584831 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584786 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-socket\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584831 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584815 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-certs\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.584986 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.584895 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-credential-socket\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.585194 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.585177 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istiod-ca-cert\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.586751 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.586725 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-envoy\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.586888 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.586872 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-podinfo\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.592348 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.592287 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-token\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.592461 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.592392 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn9tg\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-kube-api-access-fn9tg\") pod \"e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.656459 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.656422 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:43.801148 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.801123 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5"] Jul 02 22:12:43.802012 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:12:43.801984 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c5bbd67_e155_4aef_b66b_75a1d7b6f348.slice/crio-3a99dc6fccece6a87a5a99786f7290440fbf3246644021f1393e15afaab84a08 WatchSource:0}: Error finding container 3a99dc6fccece6a87a5a99786f7290440fbf3246644021f1393e15afaab84a08: Status 404 returned error can't find the container with id 3a99dc6fccece6a87a5a99786f7290440fbf3246644021f1393e15afaab84a08 Jul 02 22:12:43.807562 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.807526 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:43.807662 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.807591 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:43.807662 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:43.807633 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:44.741510 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:44.741474 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" event={"ID":"8c5bbd67-e155-4aef-b66b-75a1d7b6f348","Type":"ContainerStarted","Data":"696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df"} Jul 02 22:12:44.741906 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:44.741528 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" event={"ID":"8c5bbd67-e155-4aef-b66b-75a1d7b6f348","Type":"ContainerStarted","Data":"3a99dc6fccece6a87a5a99786f7290440fbf3246644021f1393e15afaab84a08"} Jul 02 22:12:44.762560 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:44.762515 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" podStartSLOduration=1.762500018 podStartE2EDuration="1.762500018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:44.76089434 +0000 UTC m=+2499.210732975" watchObservedRunningTime="2026-07-02 22:12:44.762500018 +0000 UTC m=+2499.212338643" Jul 02 22:12:45.657257 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:45.657223 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:45.662783 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:45.662754 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:45.748088 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:45.748055 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:45.749141 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:45.749122 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:12:47.751187 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.751122 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:47.762509 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.762481 2547 generic.go:356] "Generic (PLEG): container finished" podID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerID="95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33" exitCode=0 Jul 02 22:12:47.762697 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.762535 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" Jul 02 22:12:47.762697 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.762563 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" event={"ID":"9c64fe6e-b081-44fe-9f80-1e6e36d265a9","Type":"ContainerDied","Data":"95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33"} Jul 02 22:12:47.762697 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.762627 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b" event={"ID":"9c64fe6e-b081-44fe-9f80-1e6e36d265a9","Type":"ContainerDied","Data":"4679e81680ef983934b1c111e29a8b1040775bc3a0ffdf95babf8a66a68062ca"} Jul 02 22:12:47.762697 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.762658 2547 scope.go:117] "RemoveContainer" containerID="95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33" Jul 02 22:12:47.773161 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.772821 2547 scope.go:117] "RemoveContainer" containerID="94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c" Jul 02 22:12:47.784719 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.784694 2547 scope.go:117] "RemoveContainer" containerID="95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33" Jul 02 22:12:47.785054 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:12:47.785034 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33\": container with ID starting with 95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33 not found: ID does not exist" containerID="95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33" Jul 02 22:12:47.785132 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.785064 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33"} err="failed to get container status \"95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33\": rpc error: code = NotFound desc = could not find container \"95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33\": container with ID starting with 95725f2c2ec1186ad65954c2156c71ddfcccb24c81295c5c10ecf676e5cebf33 not found: ID does not exist" Jul 02 22:12:47.785132 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.785092 2547 scope.go:117] "RemoveContainer" containerID="94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c" Jul 02 22:12:47.785364 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:12:47.785344 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c\": container with ID starting with 94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c not found: ID does not exist" containerID="94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c" Jul 02 22:12:47.785520 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.785374 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c"} err="failed to get container status \"94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c\": rpc error: code = NotFound desc = could not find container \"94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c\": container with ID starting with 94438a2c24ef98dffb47f59f234198863dabba3a08699dca01d2250e77921e0c not found: ID does not exist" Jul 02 22:12:47.831531 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831500 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-certs\") pod \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " Jul 02 22:12:47.831531 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831536 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-socket\") pod \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " Jul 02 22:12:47.831776 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831592 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-envoy\") pod \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " Jul 02 22:12:47.831776 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831637 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-podinfo\") pod \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " Jul 02 22:12:47.831776 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831674 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istiod-ca-cert\") pod \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " Jul 02 22:12:47.831776 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831708 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhcn4\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-kube-api-access-mhcn4\") pod \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " Jul 02 22:12:47.831776 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831745 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-credential-socket\") pod \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " Jul 02 22:12:47.832033 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831786 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-token\") pod \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " Jul 02 22:12:47.832033 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831815 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-data\") pod \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\" (UID: \"9c64fe6e-b081-44fe-9f80-1e6e36d265a9\") " Jul 02 22:12:47.832033 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831821 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9c64fe6e-b081-44fe-9f80-1e6e36d265a9" (UID: "9c64fe6e-b081-44fe-9f80-1e6e36d265a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:47.832033 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.831861 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9c64fe6e-b081-44fe-9f80-1e6e36d265a9" (UID: "9c64fe6e-b081-44fe-9f80-1e6e36d265a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:47.832247 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.832051 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9c64fe6e-b081-44fe-9f80-1e6e36d265a9" (UID: "9c64fe6e-b081-44fe-9f80-1e6e36d265a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:47.832247 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.832167 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9c64fe6e-b081-44fe-9f80-1e6e36d265a9" (UID: "9c64fe6e-b081-44fe-9f80-1e6e36d265a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:12:47.832341 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.832328 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:47.832417 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.832349 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:47.832417 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.832366 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:47.832515 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.832420 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:47.832697 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.832674 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "9c64fe6e-b081-44fe-9f80-1e6e36d265a9" (UID: "9c64fe6e-b081-44fe-9f80-1e6e36d265a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:47.834069 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.834043 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9c64fe6e-b081-44fe-9f80-1e6e36d265a9" (UID: "9c64fe6e-b081-44fe-9f80-1e6e36d265a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:12:47.834177 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.834158 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-kube-api-access-mhcn4" (OuterVolumeSpecName: "kube-api-access-mhcn4") pod "9c64fe6e-b081-44fe-9f80-1e6e36d265a9" (UID: "9c64fe6e-b081-44fe-9f80-1e6e36d265a9"). InnerVolumeSpecName "kube-api-access-mhcn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:47.834313 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.834292 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c64fe6e-b081-44fe-9f80-1e6e36d265a9" (UID: "9c64fe6e-b081-44fe-9f80-1e6e36d265a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:12:47.834373 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.834317 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9c64fe6e-b081-44fe-9f80-1e6e36d265a9" (UID: "9c64fe6e-b081-44fe-9f80-1e6e36d265a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:12:47.933842 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.933806 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:47.933842 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.933836 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:47.933842 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.933848 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhcn4\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-kube-api-access-mhcn4\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:47.934129 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.933860 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:47.934129 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:47.933874 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c64fe6e-b081-44fe-9f80-1e6e36d265a9-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:12:48.079666 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:48.079629 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b"] Jul 02 22:12:48.085882 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:48.085855 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-6f0ab8-openshift-default-5f89d8b987-65x8b"] Jul 02 22:12:50.009451 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:50.009420 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" path="/var/lib/kubelet/pods/9c64fe6e-b081-44fe-9f80-1e6e36d265a9/volumes" Jul 02 22:12:51.816998 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.816967 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl"] Jul 02 22:12:51.817669 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.817649 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" Jul 02 22:12:51.817669 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.817671 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" Jul 02 22:12:51.817834 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.817798 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" Jul 02 22:12:51.817834 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.817818 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee1f990-34c6-460b-aa1c-de3950bf4e9c" containerName="istio-proxy" Jul 02 22:12:51.817834 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.817828 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" Jul 02 22:12:51.822756 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.822730 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:51.825418 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.825366 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a51408-openshift-default-dockercfg-zck9k\"" Jul 02 22:12:51.833502 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.833478 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl"] Jul 02 22:12:51.974964 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.974917 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-podinfo\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:51.975282 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.975259 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istiod-ca-cert\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:51.975458 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.975442 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-socket\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:51.975667 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.975633 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-certs\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:51.975858 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.975841 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-token\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:51.976957 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.976938 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-envoy\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:51.977136 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.977119 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-data\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:51.977278 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.977261 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw96h\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-kube-api-access-hw96h\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:51.977414 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:51.977399 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-credential-socket\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.078531 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.078416 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-envoy\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.078531 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.078505 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-data\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.078785 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.078535 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw96h\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-kube-api-access-hw96h\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.078785 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.078577 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-credential-socket\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.078785 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.078610 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-podinfo\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.078785 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.078675 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istiod-ca-cert\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.078785 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.078724 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-socket\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.078785 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.078781 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-certs\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.079102 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.078811 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-token\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.080268 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.080240 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-socket\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.080926 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.080857 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istiod-ca-cert\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.081360 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.081336 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-credential-socket\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.081677 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.081657 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-data\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.081800 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.081347 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-certs\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.082595 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.082566 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-podinfo\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.082695 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.082652 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-envoy\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.093728 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.093697 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw96h\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-kube-api-access-hw96h\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.095169 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.095114 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-token\") pod \"e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.137429 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.137364 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:52.334931 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.334905 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl"] Jul 02 22:12:52.336263 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:12:52.336235 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60272dcb_4ec4_4402_971a_3fe0d2b9ee3a.slice/crio-3b65f04f0ed83e50f052cf8f087c96f9734f56aa90bd1741c56b082f43571c75 WatchSource:0}: Error finding container 3b65f04f0ed83e50f052cf8f087c96f9734f56aa90bd1741c56b082f43571c75: Status 404 returned error can't find the container with id 3b65f04f0ed83e50f052cf8f087c96f9734f56aa90bd1741c56b082f43571c75 Jul 02 22:12:52.338413 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.338359 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:52.338510 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.338449 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:52.338510 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.338478 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:52.799300 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.799260 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" event={"ID":"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a","Type":"ContainerStarted","Data":"cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5"} Jul 02 22:12:52.799496 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.799325 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" event={"ID":"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a","Type":"ContainerStarted","Data":"3b65f04f0ed83e50f052cf8f087c96f9734f56aa90bd1741c56b082f43571c75"} Jul 02 22:12:52.822211 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:52.822161 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" podStartSLOduration=1.8221454609999999 podStartE2EDuration="1.822145461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:12:52.82009022 +0000 UTC m=+2507.269928847" watchObservedRunningTime="2026-07-02 22:12:52.822145461 +0000 UTC m=+2507.271984084" Jul 02 22:12:53.137757 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:53.137677 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:53.142997 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:53.142972 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:53.808587 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:53.808563 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:53.809511 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:53.809487 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:12:59.846196 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:59.846164 2547 generic.go:356] "Generic (PLEG): container finished" podID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerID="696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df" exitCode=0 Jul 02 22:12:59.846642 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:59.846242 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" event={"ID":"8c5bbd67-e155-4aef-b66b-75a1d7b6f348","Type":"ContainerDied","Data":"696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df"} Jul 02 22:12:59.846709 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:59.846688 2547 scope.go:117] "RemoveContainer" containerID="696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df" Jul 02 22:12:59.847833 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:59.847808 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:59.847892 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:59.847871 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:12:59.847928 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:12:59.847904 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:13:00.748461 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:00.748431 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:13:00.854758 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:00.854716 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" event={"ID":"8c5bbd67-e155-4aef-b66b-75a1d7b6f348","Type":"ContainerStarted","Data":"82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114"} Jul 02 22:13:01.656853 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:01.656806 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:13:01.658224 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:01.658192 2547 patch_prober.go:28] interesting pod/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Jul 02 22:13:01.658364 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:01.658256 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Jul 02 22:13:02.657285 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:02.657241 2547 patch_prober.go:28] interesting pod/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Jul 02 22:13:02.657821 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:02.657310 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Jul 02 22:13:03.657880 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:03.657838 2547 patch_prober.go:28] interesting pod/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Jul 02 22:13:03.658418 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:03.657904 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Jul 02 22:13:04.657398 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:04.657339 2547 patch_prober.go:28] interesting pod/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Jul 02 22:13:04.657592 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:04.657456 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Jul 02 22:13:05.661260 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:05.661230 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:13:05.661735 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:05.661658 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:13:05.662302 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:05.662285 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:13:09.425085 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:09.424272 2547 patch_prober.go:28] interesting pod/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": read tcp 10.132.0.2:40752->10.132.0.78:15021: read: connection reset by peer" start-of-body= Jul 02 22:13:09.425085 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:09.424338 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": read tcp 10.132.0.2:40752->10.132.0.78:15021: read: connection reset by peer" Jul 02 22:13:09.923737 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:09.923705 2547 generic.go:356] "Generic (PLEG): container finished" podID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerID="cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5" exitCode=0 Jul 02 22:13:09.923921 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:09.923775 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" event={"ID":"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a","Type":"ContainerDied","Data":"cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5"} Jul 02 22:13:09.924207 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:09.924194 2547 scope.go:117] "RemoveContainer" containerID="cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5" Jul 02 22:13:09.925211 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:09.925182 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:13:09.925285 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:09.925252 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:13:09.925328 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:09.925286 2547 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 02 22:13:10.931556 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:10.931519 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" event={"ID":"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a","Type":"ContainerStarted","Data":"df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c"} Jul 02 22:13:11.138215 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:11.138178 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:13:11.138474 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:11.138451 2547 patch_prober.go:28] interesting pod/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Jul 02 22:13:11.138527 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:11.138500 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Jul 02 22:13:12.138056 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:12.138023 2547 patch_prober.go:28] interesting pod/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Jul 02 22:13:12.138559 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:12.138080 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Jul 02 22:13:13.137701 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:13.137666 2547 patch_prober.go:28] interesting pod/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Jul 02 22:13:13.137874 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:13.137734 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Jul 02 22:13:14.138558 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:14.138525 2547 patch_prober.go:28] interesting pod/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Jul 02 22:13:14.138942 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:14.138575 2547 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Jul 02 22:13:15.148312 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:15.148275 2547 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:13:15.148770 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:15.148676 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:13:15.149431 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:13:15.149410 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:14:06.309341 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:14:06.309307 2547 scope.go:117] "RemoveContainer" containerID="139d604a53b71130d5846b192c67b8007abc476c653ffbdf43eb68576f6ae51a" Jul 02 22:15:00.134159 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.134127 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn"] Jul 02 22:15:00.134897 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.134878 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" Jul 02 22:15:00.134943 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.134900 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c64fe6e-b081-44fe-9f80-1e6e36d265a9" containerName="istio-proxy" Jul 02 22:15:00.138744 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.138721 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29717175-7kvmz"] Jul 02 22:15:00.138873 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.138826 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" Jul 02 22:15:00.141269 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.141244 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-vnrlr\"" Jul 02 22:15:00.143147 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.143106 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6"] Jul 02 22:15:00.143585 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.143566 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" Jul 02 22:15:00.147788 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.147762 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6"] Jul 02 22:15:00.147911 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.147871 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" Jul 02 22:15:00.150593 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.148992 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn"] Jul 02 22:15:00.153003 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.152983 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717175-7kvmz"] Jul 02 22:15:00.200944 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.200910 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq7xm\" (UniqueName: \"kubernetes.io/projected/70cb5ba9-6379-4db5-9b41-c3f22481f235-kube-api-access-vq7xm\") pod \"maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn\" (UID: \"70cb5ba9-6379-4db5-9b41-c3f22481f235\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" Jul 02 22:15:00.201135 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.200956 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbsvm\" (UniqueName: \"kubernetes.io/projected/f26ebcd9-6645-444a-aea7-639d3b513f70-kube-api-access-pbsvm\") pod \"maas-api-key-cleanup-29717175-7kvmz\" (UID: \"f26ebcd9-6645-444a-aea7-639d3b513f70\") " pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" Jul 02 22:15:00.201135 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.200978 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwvff\" (UniqueName: \"kubernetes.io/projected/1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6-kube-api-access-rwvff\") pod \"maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6\" (UID: \"1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" Jul 02 22:15:00.302084 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.302044 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbsvm\" (UniqueName: \"kubernetes.io/projected/f26ebcd9-6645-444a-aea7-639d3b513f70-kube-api-access-pbsvm\") pod \"maas-api-key-cleanup-29717175-7kvmz\" (UID: \"f26ebcd9-6645-444a-aea7-639d3b513f70\") " pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" Jul 02 22:15:00.302084 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.302086 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwvff\" (UniqueName: \"kubernetes.io/projected/1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6-kube-api-access-rwvff\") pod \"maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6\" (UID: \"1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" Jul 02 22:15:00.302360 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.302194 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq7xm\" (UniqueName: \"kubernetes.io/projected/70cb5ba9-6379-4db5-9b41-c3f22481f235-kube-api-access-vq7xm\") pod \"maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn\" (UID: \"70cb5ba9-6379-4db5-9b41-c3f22481f235\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" Jul 02 22:15:00.310996 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.310963 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbsvm\" (UniqueName: \"kubernetes.io/projected/f26ebcd9-6645-444a-aea7-639d3b513f70-kube-api-access-pbsvm\") pod \"maas-api-key-cleanup-29717175-7kvmz\" (UID: \"f26ebcd9-6645-444a-aea7-639d3b513f70\") " pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" Jul 02 22:15:00.311648 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.311618 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq7xm\" (UniqueName: \"kubernetes.io/projected/70cb5ba9-6379-4db5-9b41-c3f22481f235-kube-api-access-vq7xm\") pod \"maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn\" (UID: \"70cb5ba9-6379-4db5-9b41-c3f22481f235\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" Jul 02 22:15:00.311778 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.311754 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwvff\" (UniqueName: \"kubernetes.io/projected/1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6-kube-api-access-rwvff\") pod \"maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6\" (UID: \"1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" Jul 02 22:15:00.455186 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.455100 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" Jul 02 22:15:00.464411 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.464373 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" Jul 02 22:15:00.471466 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.471440 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" Jul 02 22:15:00.834910 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.834888 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn"] Jul 02 22:15:00.835792 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:15:00.835772 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70cb5ba9_6379_4db5_9b41_c3f22481f235.slice/crio-71095b8be6db359e682bb086fd5a1317f3ed0b5facce432755e00813a9ec211e WatchSource:0}: Error finding container 71095b8be6db359e682bb086fd5a1317f3ed0b5facce432755e00813a9ec211e: Status 404 returned error can't find the container with id 71095b8be6db359e682bb086fd5a1317f3ed0b5facce432755e00813a9ec211e Jul 02 22:15:00.868507 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.868479 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6"] Jul 02 22:15:00.869911 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:15:00.869868 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5e2d5f_ed1e_4505_84d6_3a6bae722ce6.slice/crio-bb5b156d159b1c91ac75c9dd7cedea5800e9902a3ad87ef835b3c00619f08ec3 WatchSource:0}: Error finding container bb5b156d159b1c91ac75c9dd7cedea5800e9902a3ad87ef835b3c00619f08ec3: Status 404 returned error can't find the container with id bb5b156d159b1c91ac75c9dd7cedea5800e9902a3ad87ef835b3c00619f08ec3 Jul 02 22:15:00.872809 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:00.872766 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29717175-7kvmz"] Jul 02 22:15:00.872931 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:15:00.872909 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26ebcd9_6645_444a_aea7_639d3b513f70.slice/crio-d47f50a3772377d781585c1bbb7740abe14cdbc6b3ea7d15dd13c6b54dc46d28 WatchSource:0}: Error finding container d47f50a3772377d781585c1bbb7740abe14cdbc6b3ea7d15dd13c6b54dc46d28: Status 404 returned error can't find the container with id d47f50a3772377d781585c1bbb7740abe14cdbc6b3ea7d15dd13c6b54dc46d28 Jul 02 22:15:01.601673 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:01.601635 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" event={"ID":"1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6","Type":"ContainerStarted","Data":"19d6bc73f460c482443203973af7f11b81989d361df478ceebad920faf0a6505"} Jul 02 22:15:01.602108 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:01.601701 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" event={"ID":"1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6","Type":"ContainerStarted","Data":"bb5b156d159b1c91ac75c9dd7cedea5800e9902a3ad87ef835b3c00619f08ec3"} Jul 02 22:15:01.603247 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:01.603224 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" event={"ID":"70cb5ba9-6379-4db5-9b41-c3f22481f235","Type":"ContainerStarted","Data":"b2e0078b079a01360f3d6c764cb2193b1b42a10394bac20ee7103cfa0622fe74"} Jul 02 22:15:01.603341 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:01.603273 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" event={"ID":"70cb5ba9-6379-4db5-9b41-c3f22481f235","Type":"ContainerStarted","Data":"71095b8be6db359e682bb086fd5a1317f3ed0b5facce432755e00813a9ec211e"} Jul 02 22:15:01.604684 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:01.604663 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" event={"ID":"f26ebcd9-6645-444a-aea7-639d3b513f70","Type":"ContainerStarted","Data":"a37fbcc56dadb62688bd7cb8d205a5f46d7ef7628c1cab5d3e931e3a1cc9287c"} Jul 02 22:15:01.604768 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:01.604699 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" event={"ID":"f26ebcd9-6645-444a-aea7-639d3b513f70","Type":"ContainerStarted","Data":"d47f50a3772377d781585c1bbb7740abe14cdbc6b3ea7d15dd13c6b54dc46d28"} Jul 02 22:15:01.618664 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:01.618622 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" podStartSLOduration=1.618608649 podStartE2EDuration="1.618608649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:15:01.616872248 +0000 UTC m=+2636.066710877" watchObservedRunningTime="2026-07-02 22:15:01.618608649 +0000 UTC m=+2636.068447273" Jul 02 22:15:01.633515 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:01.633472 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" podStartSLOduration=1.633457919 podStartE2EDuration="1.633457919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:15:01.632017435 +0000 UTC m=+2636.081856061" watchObservedRunningTime="2026-07-02 22:15:01.633457919 +0000 UTC m=+2636.083296584" Jul 02 22:15:01.649445 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:01.649367 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" podStartSLOduration=1.6493518219999999 podStartE2EDuration="1.649351822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:15:01.64685463 +0000 UTC m=+2636.096693255" watchObservedRunningTime="2026-07-02 22:15:01.649351822 +0000 UTC m=+2636.099190446" Jul 02 22:15:02.612894 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:02.612855 2547 generic.go:356] "Generic (PLEG): container finished" podID="70cb5ba9-6379-4db5-9b41-c3f22481f235" containerID="b2e0078b079a01360f3d6c764cb2193b1b42a10394bac20ee7103cfa0622fe74" exitCode=0 Jul 02 22:15:02.613328 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:02.612899 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" event={"ID":"70cb5ba9-6379-4db5-9b41-c3f22481f235","Type":"ContainerDied","Data":"b2e0078b079a01360f3d6c764cb2193b1b42a10394bac20ee7103cfa0622fe74"} Jul 02 22:15:02.614604 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:02.614583 2547 generic.go:356] "Generic (PLEG): container finished" podID="f26ebcd9-6645-444a-aea7-639d3b513f70" containerID="a37fbcc56dadb62688bd7cb8d205a5f46d7ef7628c1cab5d3e931e3a1cc9287c" exitCode=0 Jul 02 22:15:02.614699 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:02.614625 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" event={"ID":"f26ebcd9-6645-444a-aea7-639d3b513f70","Type":"ContainerDied","Data":"a37fbcc56dadb62688bd7cb8d205a5f46d7ef7628c1cab5d3e931e3a1cc9287c"} Jul 02 22:15:02.616231 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:02.616211 2547 generic.go:356] "Generic (PLEG): container finished" podID="1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6" containerID="19d6bc73f460c482443203973af7f11b81989d361df478ceebad920faf0a6505" exitCode=0 Jul 02 22:15:02.616342 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:02.616256 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" event={"ID":"1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6","Type":"ContainerDied","Data":"19d6bc73f460c482443203973af7f11b81989d361df478ceebad920faf0a6505"} Jul 02 22:15:03.789159 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:03.789122 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" Jul 02 22:15:03.837111 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:03.837087 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" Jul 02 22:15:03.840610 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:03.840590 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" Jul 02 22:15:03.939366 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:03.939253 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vq7xm\" (UniqueName: \"kubernetes.io/projected/70cb5ba9-6379-4db5-9b41-c3f22481f235-kube-api-access-vq7xm\") pod \"70cb5ba9-6379-4db5-9b41-c3f22481f235\" (UID: \"70cb5ba9-6379-4db5-9b41-c3f22481f235\") " Jul 02 22:15:03.939578 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:03.939372 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwvff\" (UniqueName: \"kubernetes.io/projected/1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6-kube-api-access-rwvff\") pod \"1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6\" (UID: \"1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6\") " Jul 02 22:15:03.939578 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:03.939436 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbsvm\" (UniqueName: \"kubernetes.io/projected/f26ebcd9-6645-444a-aea7-639d3b513f70-kube-api-access-pbsvm\") pod \"f26ebcd9-6645-444a-aea7-639d3b513f70\" (UID: \"f26ebcd9-6645-444a-aea7-639d3b513f70\") " Jul 02 22:15:03.941593 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:03.941562 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f26ebcd9-6645-444a-aea7-639d3b513f70-kube-api-access-pbsvm" (OuterVolumeSpecName: "kube-api-access-pbsvm") pod "f26ebcd9-6645-444a-aea7-639d3b513f70" (UID: "f26ebcd9-6645-444a-aea7-639d3b513f70"). InnerVolumeSpecName "kube-api-access-pbsvm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:15:03.941593 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:03.941574 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70cb5ba9-6379-4db5-9b41-c3f22481f235-kube-api-access-vq7xm" (OuterVolumeSpecName: "kube-api-access-vq7xm") pod "70cb5ba9-6379-4db5-9b41-c3f22481f235" (UID: "70cb5ba9-6379-4db5-9b41-c3f22481f235"). InnerVolumeSpecName "kube-api-access-vq7xm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:15:03.941744 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:03.941603 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6-kube-api-access-rwvff" (OuterVolumeSpecName: "kube-api-access-rwvff") pod "1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6" (UID: "1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6"). InnerVolumeSpecName "kube-api-access-rwvff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:15:04.041162 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.041135 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwvff\" (UniqueName: \"kubernetes.io/projected/1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6-kube-api-access-rwvff\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:15:04.041162 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.041161 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbsvm\" (UniqueName: \"kubernetes.io/projected/f26ebcd9-6645-444a-aea7-639d3b513f70-kube-api-access-pbsvm\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:15:04.041318 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.041172 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vq7xm\" (UniqueName: \"kubernetes.io/projected/70cb5ba9-6379-4db5-9b41-c3f22481f235-kube-api-access-vq7xm\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:15:04.629437 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.629405 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" Jul 02 22:15:04.629623 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.629412 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn" event={"ID":"70cb5ba9-6379-4db5-9b41-c3f22481f235","Type":"ContainerDied","Data":"71095b8be6db359e682bb086fd5a1317f3ed0b5facce432755e00813a9ec211e"} Jul 02 22:15:04.629623 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.629558 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="71095b8be6db359e682bb086fd5a1317f3ed0b5facce432755e00813a9ec211e" Jul 02 22:15:04.630815 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.630794 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" Jul 02 22:15:04.630815 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.630804 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29717175-7kvmz" event={"ID":"f26ebcd9-6645-444a-aea7-639d3b513f70","Type":"ContainerDied","Data":"d47f50a3772377d781585c1bbb7740abe14cdbc6b3ea7d15dd13c6b54dc46d28"} Jul 02 22:15:04.631022 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.630843 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d47f50a3772377d781585c1bbb7740abe14cdbc6b3ea7d15dd13c6b54dc46d28" Jul 02 22:15:04.632282 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.632265 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" Jul 02 22:15:04.632282 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.632270 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6" event={"ID":"1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6","Type":"ContainerDied","Data":"bb5b156d159b1c91ac75c9dd7cedea5800e9902a3ad87ef835b3c00619f08ec3"} Jul 02 22:15:04.632490 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:15:04.632309 2547 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb5b156d159b1c91ac75c9dd7cedea5800e9902a3ad87ef835b3c00619f08ec3" Jul 02 22:16:06.192680 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:16:06.192651 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 22:16:06.200182 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:16:06.200156 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 22:16:06.202607 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:16:06.202589 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 22:16:06.209403 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:16:06.209373 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-84htm_818e1edd-f50a-43b0-86a5-03c2c318e468/ovn-acl-logging/0.log" Jul 02 22:18:36.214602 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:18:36.214567 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn"] Jul 02 22:18:36.217420 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:18:36.217392 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-77374e-29717175-w8xcn"] Jul 02 22:18:38.009230 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:18:38.009196 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70cb5ba9-6379-4db5-9b41-c3f22481f235" path="/var/lib/kubelet/pods/70cb5ba9-6379-4db5-9b41-c3f22481f235/volumes" Jul 02 22:19:17.948163 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:17.948127 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5"] Jul 02 22:19:17.948668 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:17.948404 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" containerID="cri-o://82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114" gracePeriod=30 Jul 02 22:19:19.022720 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:19.022682 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6"] Jul 02 22:19:19.025584 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:19.025557 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-a51408-29717175-wq5x6"] Jul 02 22:19:20.008877 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:20.008844 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6" path="/var/lib/kubelet/pods/1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6/volumes" Jul 02 22:19:20.664440 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:20.664403 2547 patch_prober.go:28] interesting pod/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 02 22:19:20.664825 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:20.664464 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 22:19:23.214109 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.214050 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:19:23.263333 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.263296 2547 generic.go:356] "Generic (PLEG): container finished" podID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerID="82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114" exitCode=0 Jul 02 22:19:23.263502 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.263419 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" Jul 02 22:19:23.263502 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.263430 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" event={"ID":"8c5bbd67-e155-4aef-b66b-75a1d7b6f348","Type":"ContainerDied","Data":"82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114"} Jul 02 22:19:23.263502 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.263489 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5" event={"ID":"8c5bbd67-e155-4aef-b66b-75a1d7b6f348","Type":"ContainerDied","Data":"3a99dc6fccece6a87a5a99786f7290440fbf3246644021f1393e15afaab84a08"} Jul 02 22:19:23.263610 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.263521 2547 scope.go:117] "RemoveContainer" containerID="82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114" Jul 02 22:19:23.273157 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.273139 2547 scope.go:117] "RemoveContainer" containerID="696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df" Jul 02 22:19:23.281507 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.281490 2547 scope.go:117] "RemoveContainer" containerID="82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114" Jul 02 22:19:23.281742 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:19:23.281722 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114\": container with ID starting with 82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114 not found: ID does not exist" containerID="82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114" Jul 02 22:19:23.281792 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.281755 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114"} err="failed to get container status \"82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114\": rpc error: code = NotFound desc = could not find container \"82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114\": container with ID starting with 82edb9e0d09fd86f3bfecef50536b1e40eb1445bd369258b02b0477e03551114 not found: ID does not exist" Jul 02 22:19:23.281792 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.281777 2547 scope.go:117] "RemoveContainer" containerID="696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df" Jul 02 22:19:23.282018 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:19:23.282003 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df\": container with ID starting with 696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df not found: ID does not exist" containerID="696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df" Jul 02 22:19:23.282059 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.282021 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df"} err="failed to get container status \"696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df\": rpc error: code = NotFound desc = could not find container \"696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df\": container with ID starting with 696903f895df634ca225cc9ebb82187756545d5778af42d34be0fa96e8b4a3df not found: ID does not exist" Jul 02 22:19:23.300547 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300524 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istiod-ca-cert\") pod \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " Jul 02 22:19:23.300634 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300552 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fn9tg\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-kube-api-access-fn9tg\") pod \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " Jul 02 22:19:23.300634 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300583 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-certs\") pod \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " Jul 02 22:19:23.300634 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300613 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-podinfo\") pod \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " Jul 02 22:19:23.300742 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300642 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-envoy\") pod \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " Jul 02 22:19:23.300742 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300668 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-data\") pod \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " Jul 02 22:19:23.300830 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300812 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-token\") pod \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " Jul 02 22:19:23.300885 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300830 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c5bbd67-e155-4aef-b66b-75a1d7b6f348" (UID: "8c5bbd67-e155-4aef-b66b-75a1d7b6f348"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:23.300885 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300871 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-credential-socket\") pod \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " Jul 02 22:19:23.300994 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300925 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-socket\") pod \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\" (UID: \"8c5bbd67-e155-4aef-b66b-75a1d7b6f348\") " Jul 02 22:19:23.300994 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.300936 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c5bbd67-e155-4aef-b66b-75a1d7b6f348" (UID: "8c5bbd67-e155-4aef-b66b-75a1d7b6f348"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:19:23.301167 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.301136 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c5bbd67-e155-4aef-b66b-75a1d7b6f348" (UID: "8c5bbd67-e155-4aef-b66b-75a1d7b6f348"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:23.301336 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.301307 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c5bbd67-e155-4aef-b66b-75a1d7b6f348" (UID: "8c5bbd67-e155-4aef-b66b-75a1d7b6f348"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:23.301491 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.301455 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:23.301491 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.301487 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:23.301592 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.301502 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:23.301592 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.301515 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:23.301659 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.301611 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c5bbd67-e155-4aef-b66b-75a1d7b6f348" (UID: "8c5bbd67-e155-4aef-b66b-75a1d7b6f348"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:23.302763 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.302730 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c5bbd67-e155-4aef-b66b-75a1d7b6f348" (UID: "8c5bbd67-e155-4aef-b66b-75a1d7b6f348"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:23.302930 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.302916 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c5bbd67-e155-4aef-b66b-75a1d7b6f348" (UID: "8c5bbd67-e155-4aef-b66b-75a1d7b6f348"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:19:23.303222 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.303203 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-kube-api-access-fn9tg" (OuterVolumeSpecName: "kube-api-access-fn9tg") pod "8c5bbd67-e155-4aef-b66b-75a1d7b6f348" (UID: "8c5bbd67-e155-4aef-b66b-75a1d7b6f348"). InnerVolumeSpecName "kube-api-access-fn9tg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:19:23.303222 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.303212 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c5bbd67-e155-4aef-b66b-75a1d7b6f348" (UID: "8c5bbd67-e155-4aef-b66b-75a1d7b6f348"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:19:23.402741 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.402716 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn9tg\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-kube-api-access-fn9tg\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:23.402741 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.402739 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:23.402914 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.402748 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:23.402914 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.402756 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:23.402914 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.402764 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c5bbd67-e155-4aef-b66b-75a1d7b6f348-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:23.587789 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.587753 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5"] Jul 02 22:19:23.594050 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:23.594026 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-77374e-openshift-default-859986fc67-jrmp5"] Jul 02 22:19:24.010780 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:24.010704 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" path="/var/lib/kubelet/pods/8c5bbd67-e155-4aef-b66b-75a1d7b6f348/volumes" Jul 02 22:19:25.011468 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:25.011438 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl"] Jul 02 22:19:25.011873 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:25.011619 2547 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" containerID="cri-o://df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c" gracePeriod=30 Jul 02 22:19:30.149725 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.149683 2547 patch_prober.go:28] interesting pod/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Jul 02 22:19:30.150300 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.149746 2547 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Jul 02 22:19:30.199129 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.199102 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jxqlb_69be0910-f104-433e-a97d-43ab24d5cd82/manager/0.log" Jul 02 22:19:30.274904 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.274882 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:19:30.310593 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.310556 2547 generic.go:356] "Generic (PLEG): container finished" podID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerID="df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c" exitCode=0 Jul 02 22:19:30.310738 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.310618 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" event={"ID":"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a","Type":"ContainerDied","Data":"df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c"} Jul 02 22:19:30.310738 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.310668 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" event={"ID":"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a","Type":"ContainerDied","Data":"3b65f04f0ed83e50f052cf8f087c96f9734f56aa90bd1741c56b082f43571c75"} Jul 02 22:19:30.310738 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.310689 2547 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl" Jul 02 22:19:30.310738 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.310702 2547 scope.go:117] "RemoveContainer" containerID="df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c" Jul 02 22:19:30.323506 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.323486 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7vbvm_ced64b90-2bad-4247-afd2-f4776cb690fd/manager/0.log" Jul 02 22:19:30.323661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.323634 2547 scope.go:117] "RemoveContainer" containerID="cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5" Jul 02 22:19:30.339360 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.339301 2547 scope.go:117] "RemoveContainer" containerID="df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c" Jul 02 22:19:30.339656 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:19:30.339634 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c\": container with ID starting with df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c not found: ID does not exist" containerID="df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c" Jul 02 22:19:30.339736 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.339669 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c"} err="failed to get container status \"df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c\": rpc error: code = NotFound desc = could not find container \"df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c\": container with ID starting with df70420884cc3914a7bbfa2f2842dfd29b0f60ff3c5999ff2a5fe24a84ac7c0c not found: ID does not exist" Jul 02 22:19:30.339736 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.339692 2547 scope.go:117] "RemoveContainer" containerID="cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5" Jul 02 22:19:30.339938 ip-10-0-137-35 kubenswrapper[2547]: E0702 22:19:30.339920 2547 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5\": container with ID starting with cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5 not found: ID does not exist" containerID="cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5" Jul 02 22:19:30.339996 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.339948 2547 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5"} err="failed to get container status \"cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5\": rpc error: code = NotFound desc = could not find container \"cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5\": container with ID starting with cf7b01c6d8eb22be89b3f1c627229b27ae2734b4d97a1958948a95021b2495c5 not found: ID does not exist" Jul 02 22:19:30.371635 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371595 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-credential-socket\") pod \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " Jul 02 22:19:30.371751 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371644 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-socket\") pod \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " Jul 02 22:19:30.371751 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371676 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-podinfo\") pod \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " Jul 02 22:19:30.371751 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371697 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istiod-ca-cert\") pod \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " Jul 02 22:19:30.371751 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371742 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-certs\") pod \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " Jul 02 22:19:30.371961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371772 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw96h\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-kube-api-access-hw96h\") pod \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " Jul 02 22:19:30.371961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371835 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-token\") pod \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " Jul 02 22:19:30.371961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371869 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" (UID: "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:30.371961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371908 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-data\") pod \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " Jul 02 22:19:30.371961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371936 2547 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-envoy\") pod \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\" (UID: \"60272dcb-4ec4-4402-971a-3fe0d2b9ee3a\") " Jul 02 22:19:30.371961 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.371938 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" (UID: "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:30.372370 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.372279 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" (UID: "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 22:19:30.372370 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.372340 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" (UID: "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:30.372518 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.372440 2547 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-credential-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:30.372518 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.372480 2547 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-socket\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:30.372622 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.372521 2547 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istiod-ca-cert\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:30.372895 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.372871 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-data" (OuterVolumeSpecName: "istio-data") pod "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" (UID: "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:30.374093 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.374049 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-token" (OuterVolumeSpecName: "istio-token") pod "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" (UID: "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:19:30.374093 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.374082 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" (UID: "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 22:19:30.374351 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.374327 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-kube-api-access-hw96h" (OuterVolumeSpecName: "kube-api-access-hw96h") pod "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" (UID: "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a"). InnerVolumeSpecName "kube-api-access-hw96h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 22:19:30.374457 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.374356 2547 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" (UID: "60272dcb-4ec4-4402-971a-3fe0d2b9ee3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 02 22:19:30.473117 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.473088 2547 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-workload-certs\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:30.473117 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.473113 2547 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw96h\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-kube-api-access-hw96h\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:30.473117 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.473123 2547 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-token\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:30.473327 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.473132 2547 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-data\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:30.473327 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.473139 2547 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-envoy\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:30.473327 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.473146 2547 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a-istio-podinfo\") on node \"ip-10-0-137-35.ec2.internal\" DevicePath \"\"" Jul 02 22:19:30.555810 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.555780 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717160-2b244_ea3b61bb-b6a8-4bad-a25a-031c89bab0a2/cleanup/0.log" Jul 02 22:19:30.633827 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.633797 2547 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl"] Jul 02 22:19:30.637611 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.637583 2547 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a51408-openshift-default-857bc84fff-8gnkl"] Jul 02 22:19:30.665607 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.665585 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717175-7kvmz_f26ebcd9-6645-444a-aea7-639d3b513f70/cleanup/0.log" Jul 02 22:19:30.788565 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.788531 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-794b6f76fd-ttmvw_1a074673-6489-4874-b9e5-723117d22873/manager/0.log" Jul 02 22:19:30.896776 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:30.896694 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-t577c_02d4eac5-d07a-4f24-893a-b24c5cec4b1d/server/0.log" Jul 02 22:19:31.019184 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:31.019153 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bqp65_49cca079-af7a-44a6-b044-574a00ec939f/manager/0.log" Jul 02 22:19:31.265730 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:31.265649 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4697655-gpt2x_85d72d11-b3d1-4c52-8556-bb51acdde7c5/manager/0.log" Jul 02 22:19:31.271511 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:31.271493 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4697655-gpt2x_85d72d11-b3d1-4c52-8556-bb51acdde7c5/copy-manifests/0.log" Jul 02 22:19:31.507682 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:31.507650 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-88fd2_a548cb99-e8e8-4d1d-a196-35fbef85d57a/postgres/0.log" Jul 02 22:19:32.009794 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.009760 2547 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" path="/var/lib/kubelet/pods/60272dcb-4ec4-4402-971a-3fe0d2b9ee3a/volumes" Jul 02 22:19:32.265868 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.265785 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj_d9063a72-ba5a-4bcf-9d99-2e87829a414c/util/0.log" Jul 02 22:19:32.271958 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.271934 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj_d9063a72-ba5a-4bcf-9d99-2e87829a414c/pull/0.log" Jul 02 22:19:32.277803 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.277781 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj_d9063a72-ba5a-4bcf-9d99-2e87829a414c/extract/0.log" Jul 02 22:19:32.391418 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.391368 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt_2c4706e3-9f16-4a3d-a8b9-85e133f940d8/extract/0.log" Jul 02 22:19:32.397273 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.397252 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt_2c4706e3-9f16-4a3d-a8b9-85e133f940d8/util/0.log" Jul 02 22:19:32.404907 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.404885 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt_2c4706e3-9f16-4a3d-a8b9-85e133f940d8/pull/0.log" Jul 02 22:19:32.510705 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.510671 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh_ebb2013a-c7e2-40b0-99c2-732eb2e19b5d/util/0.log" Jul 02 22:19:32.517152 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.517087 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh_ebb2013a-c7e2-40b0-99c2-732eb2e19b5d/pull/0.log" Jul 02 22:19:32.523373 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.523353 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh_ebb2013a-c7e2-40b0-99c2-732eb2e19b5d/extract/0.log" Jul 02 22:19:32.630274 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.630247 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6_3a03b391-2b07-42c0-8970-4ccc8abb58d3/util/0.log" Jul 02 22:19:32.636742 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.636722 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6_3a03b391-2b07-42c0-8970-4ccc8abb58d3/pull/0.log" Jul 02 22:19:32.643683 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.643657 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6_3a03b391-2b07-42c0-8970-4ccc8abb58d3/extract/0.log" Jul 02 22:19:32.896087 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:32.896054 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mb7r6_2295ca83-9de0-4868-9c49-05324f1abee2/manager/0.log" Jul 02 22:19:33.009739 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:33.009712 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bfp6v_1948f8b4-17d8-473d-be76-ee7dd353e798/manager/0.log" Jul 02 22:19:33.129923 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:33.129897 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9ngp6_8ea8b1c9-5aea-44c5-a101-054d8ee19668/kuadrant-console-plugin/0.log" Jul 02 22:19:33.245965 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:33.245874 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h52vw_62cc0b04-447b-4445-a978-b3e40f5a3296/registry-server/0.log" Jul 02 22:19:33.604250 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:33.604220 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-djzzt_6ae1d647-c86f-4aaa-a478-f0dd527144c6/manager/0.log" Jul 02 22:19:33.949086 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:33.948987 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jjpt7_b3624094-3709-40d5-b87c-21645a759e14/istio-proxy/0.log" Jul 02 22:19:34.313664 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:34.313639 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d45cbf66b-c2f2f_4f9f105e-0883-4199-84ae-7c79c2f6f1dc/kube-auth-proxy/0.log" Jul 02 22:19:34.445886 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:34.445856 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-q7lbs_619e39a7-9426-48dc-a639-12103eeaead4/istio-proxy/1.log" Jul 02 22:19:34.575478 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:34.575401 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8b9x9_5730229c-9c39-4ecf-84d8-2d4579577ca3/payload-pre-processing/0.log" Jul 02 22:19:34.695035 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:34.695006 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gwgmr_2a392b5e-9307-4bc5-971c-48848af0a307/payload-processing/0.log" Jul 02 22:19:34.806814 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:34.806787 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bc4c77f4f-2bbsx_5aa921ab-8003-4f10-83ea-9842fb74856a/router/0.log" Jul 02 22:19:35.140507 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:35.140479 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qr5fw_b6a07706-b120-4b5c-b997-ae518bcf61c8/main/0.log" Jul 02 22:19:35.483805 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:35.483715 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbrl6s8_87b80306-b4d9-4a24-b1a1-c99a9137ce86/main/0.log" Jul 02 22:19:40.126420 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.126363 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-92hjf/must-gather-tzktl"] Jul 02 22:19:40.127043 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127014 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" Jul 02 22:19:40.127043 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127040 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" Jul 02 22:19:40.127249 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127053 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70cb5ba9-6379-4db5-9b41-c3f22481f235" containerName="cleanup" Jul 02 22:19:40.127249 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127062 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="70cb5ba9-6379-4db5-9b41-c3f22481f235" containerName="cleanup" Jul 02 22:19:40.127249 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127079 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f26ebcd9-6645-444a-aea7-639d3b513f70" containerName="cleanup" Jul 02 22:19:40.127249 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127088 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="f26ebcd9-6645-444a-aea7-639d3b513f70" containerName="cleanup" Jul 02 22:19:40.127249 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127099 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6" containerName="cleanup" Jul 02 22:19:40.127249 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127108 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6" containerName="cleanup" Jul 02 22:19:40.127249 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127124 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" Jul 02 22:19:40.127249 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127132 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127269 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127279 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127286 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="f26ebcd9-6645-444a-aea7-639d3b513f70" containerName="cleanup" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127292 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f5e2d5f-ed1e-4505-84d6-3a6bae722ce6" containerName="cleanup" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127297 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="70cb5ba9-6379-4db5-9b41-c3f22481f235" containerName="cleanup" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127303 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127309 2547 memory_manager.go:368] "RemoveStaleState removing state" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127368 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127373 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="60272dcb-4ec4-4402-971a-3fe0d2b9ee3a" containerName="istio-proxy" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127411 2547 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" Jul 02 22:19:40.127661 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.127417 2547 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c5bbd67-e155-4aef-b66b-75a1d7b6f348" containerName="istio-proxy" Jul 02 22:19:40.132571 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.132554 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-92hjf/must-gather-tzktl" Jul 02 22:19:40.135451 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.135433 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-92hjf\"/\"kube-root-ca.crt\"" Jul 02 22:19:40.136627 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.136611 2547 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-92hjf\"/\"openshift-service-ca.crt\"" Jul 02 22:19:40.136707 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.136617 2547 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-92hjf\"/\"default-dockercfg-m762d\"" Jul 02 22:19:40.142814 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.142791 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-92hjf/must-gather-tzktl"] Jul 02 22:19:40.268914 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.268878 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1dc3201a-4672-4a4e-9e5b-f16774eb7dbb-must-gather-output\") pod \"must-gather-tzktl\" (UID: \"1dc3201a-4672-4a4e-9e5b-f16774eb7dbb\") " pod="openshift-must-gather-92hjf/must-gather-tzktl" Jul 02 22:19:40.269116 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.268949 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzgnp\" (UniqueName: \"kubernetes.io/projected/1dc3201a-4672-4a4e-9e5b-f16774eb7dbb-kube-api-access-hzgnp\") pod \"must-gather-tzktl\" (UID: \"1dc3201a-4672-4a4e-9e5b-f16774eb7dbb\") " pod="openshift-must-gather-92hjf/must-gather-tzktl" Jul 02 22:19:40.370254 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.370222 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzgnp\" (UniqueName: \"kubernetes.io/projected/1dc3201a-4672-4a4e-9e5b-f16774eb7dbb-kube-api-access-hzgnp\") pod \"must-gather-tzktl\" (UID: \"1dc3201a-4672-4a4e-9e5b-f16774eb7dbb\") " pod="openshift-must-gather-92hjf/must-gather-tzktl" Jul 02 22:19:40.370474 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.370398 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1dc3201a-4672-4a4e-9e5b-f16774eb7dbb-must-gather-output\") pod \"must-gather-tzktl\" (UID: \"1dc3201a-4672-4a4e-9e5b-f16774eb7dbb\") " pod="openshift-must-gather-92hjf/must-gather-tzktl" Jul 02 22:19:40.370703 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.370685 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/1dc3201a-4672-4a4e-9e5b-f16774eb7dbb-must-gather-output\") pod \"must-gather-tzktl\" (UID: \"1dc3201a-4672-4a4e-9e5b-f16774eb7dbb\") " pod="openshift-must-gather-92hjf/must-gather-tzktl" Jul 02 22:19:40.378723 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.378653 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzgnp\" (UniqueName: \"kubernetes.io/projected/1dc3201a-4672-4a4e-9e5b-f16774eb7dbb-kube-api-access-hzgnp\") pod \"must-gather-tzktl\" (UID: \"1dc3201a-4672-4a4e-9e5b-f16774eb7dbb\") " pod="openshift-must-gather-92hjf/must-gather-tzktl" Jul 02 22:19:40.443518 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.443485 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-92hjf/must-gather-tzktl" Jul 02 22:19:40.597753 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.597720 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-92hjf/must-gather-tzktl"] Jul 02 22:19:40.598632 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:19:40.598600 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1dc3201a_4672_4a4e_9e5b_f16774eb7dbb.slice/crio-2cb128a88ca084dd691bc62a5e00fbce2aae222e1fcd278bdd42419cec51f0b0 WatchSource:0}: Error finding container 2cb128a88ca084dd691bc62a5e00fbce2aae222e1fcd278bdd42419cec51f0b0: Status 404 returned error can't find the container with id 2cb128a88ca084dd691bc62a5e00fbce2aae222e1fcd278bdd42419cec51f0b0 Jul 02 22:19:40.600491 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:40.600475 2547 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 22:19:41.378900 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:41.378869 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-92hjf/must-gather-tzktl" event={"ID":"1dc3201a-4672-4a4e-9e5b-f16774eb7dbb","Type":"ContainerStarted","Data":"2cb128a88ca084dd691bc62a5e00fbce2aae222e1fcd278bdd42419cec51f0b0"} Jul 02 22:19:42.390664 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:42.390627 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-92hjf/must-gather-tzktl" event={"ID":"1dc3201a-4672-4a4e-9e5b-f16774eb7dbb","Type":"ContainerStarted","Data":"f690975544a49fbfab31fa647cb7b42f041ddc2ffd7f7acc9a849e76a258b714"} Jul 02 22:19:42.390664 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:42.390679 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-92hjf/must-gather-tzktl" event={"ID":"1dc3201a-4672-4a4e-9e5b-f16774eb7dbb","Type":"ContainerStarted","Data":"eb2e068a00590504fb9bd2904b418f10b740e800374632c691e542990e31e932"} Jul 02 22:19:42.408665 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:42.408602 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-92hjf/must-gather-tzktl" podStartSLOduration=1.694863239 podStartE2EDuration="2.408577651s" totalImagesPullingTime="713.714412ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 22:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:19:42.405441938 +0000 UTC m=+2916.855280562" watchObservedRunningTime="2026-07-02 22:19:42.408577651 +0000 UTC m=+2916.858416277" Jul 02 22:19:42.926873 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:42.926838 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hsd7b_e6c4320e-5777-4358-8dac-d43f1f4f411e/global-pull-secret-syncer/0.log" Jul 02 22:19:43.062065 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:43.062025 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4z64k_6bcf0af6-b67a-4338-b2af-6431464934f1/konnectivity-agent/0.log" Jul 02 22:19:43.146816 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:43.146783 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-35.ec2.internal_c48cb01bf393d73b784190e0bda418c3/haproxy/0.log" Jul 02 22:19:46.971304 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:46.971278 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj_d9063a72-ba5a-4bcf-9d99-2e87829a414c/extract/0.log" Jul 02 22:19:47.000639 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.000608 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj_d9063a72-ba5a-4bcf-9d99-2e87829a414c/util/0.log" Jul 02 22:19:47.045910 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.045874 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759m24wj_d9063a72-ba5a-4bcf-9d99-2e87829a414c/pull/0.log" Jul 02 22:19:47.087873 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.087838 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt_2c4706e3-9f16-4a3d-a8b9-85e133f940d8/extract/0.log" Jul 02 22:19:47.118107 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.118073 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt_2c4706e3-9f16-4a3d-a8b9-85e133f940d8/util/0.log" Jul 02 22:19:47.150639 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.150605 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0dvqmt_2c4706e3-9f16-4a3d-a8b9-85e133f940d8/pull/0.log" Jul 02 22:19:47.192787 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.192755 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh_ebb2013a-c7e2-40b0-99c2-732eb2e19b5d/extract/0.log" Jul 02 22:19:47.268337 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.268258 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh_ebb2013a-c7e2-40b0-99c2-732eb2e19b5d/util/0.log" Jul 02 22:19:47.294861 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.294837 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed736wjgh_ebb2013a-c7e2-40b0-99c2-732eb2e19b5d/pull/0.log" Jul 02 22:19:47.343325 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.343283 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6_3a03b391-2b07-42c0-8970-4ccc8abb58d3/extract/0.log" Jul 02 22:19:47.365091 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.365049 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6_3a03b391-2b07-42c0-8970-4ccc8abb58d3/util/0.log" Jul 02 22:19:47.390751 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.390719 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1r58v6_3a03b391-2b07-42c0-8970-4ccc8abb58d3/pull/0.log" Jul 02 22:19:47.931474 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.931436 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mb7r6_2295ca83-9de0-4868-9c49-05324f1abee2/manager/0.log" Jul 02 22:19:47.962715 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.962674 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-bfp6v_1948f8b4-17d8-473d-be76-ee7dd353e798/manager/0.log" Jul 02 22:19:47.990567 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:47.990542 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9ngp6_8ea8b1c9-5aea-44c5-a101-054d8ee19668/kuadrant-console-plugin/0.log" Jul 02 22:19:48.055031 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:48.054996 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-h52vw_62cc0b04-447b-4445-a978-b3e40f5a3296/registry-server/0.log" Jul 02 22:19:48.363746 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:48.363710 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-djzzt_6ae1d647-c86f-4aaa-a478-f0dd527144c6/manager/0.log" Jul 02 22:19:50.097333 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.097299 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-xb5nd_64ca734d-9b7d-4802-9462-e5456cb7fb3a/cluster-monitoring-operator/0.log" Jul 02 22:19:50.130779 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.130747 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-f786h_456f8a1d-f937-458a-8063-5ebc0ee7da5b/kube-state-metrics/0.log" Jul 02 22:19:50.154160 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.154128 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-f786h_456f8a1d-f937-458a-8063-5ebc0ee7da5b/kube-rbac-proxy-main/0.log" Jul 02 22:19:50.177425 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.177392 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-f786h_456f8a1d-f937-458a-8063-5ebc0ee7da5b/kube-rbac-proxy-self/0.log" Jul 02 22:19:50.470607 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.470515 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mjz5t_ff06f541-081d-4943-89c3-00a786913e29/node-exporter/0.log" Jul 02 22:19:50.491707 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.491674 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mjz5t_ff06f541-081d-4943-89c3-00a786913e29/kube-rbac-proxy/0.log" Jul 02 22:19:50.518047 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.517985 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mjz5t_ff06f541-081d-4943-89c3-00a786913e29/init-textfile/0.log" Jul 02 22:19:50.827338 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.827255 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-cxh5x_5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9/prometheus-operator/0.log" Jul 02 22:19:50.843677 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.843608 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-cxh5x_5b3d15c9-23cf-4bb1-8a67-d23f31e6cdf9/kube-rbac-proxy/0.log" Jul 02 22:19:50.868044 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.867955 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-28k47_2b04bf6c-d4fd-4686-88ce-7c896a9bf1e9/prometheus-operator-admission-webhook/0.log" Jul 02 22:19:50.898073 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.898017 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c877d8d45-jrxrc_738a153d-aa14-4fb0-a51c-ccb26a5114c0/telemeter-client/0.log" Jul 02 22:19:50.918916 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.918890 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c877d8d45-jrxrc_738a153d-aa14-4fb0-a51c-ccb26a5114c0/reload/0.log" Jul 02 22:19:50.939336 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.939304 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-c877d8d45-jrxrc_738a153d-aa14-4fb0-a51c-ccb26a5114c0/kube-rbac-proxy/0.log" Jul 02 22:19:50.969968 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.969936 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5854f8c9-72pq4_29f1e4dd-119c-4a61-aaf1-8736217793fd/thanos-query/0.log" Jul 02 22:19:50.991230 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:50.991201 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5854f8c9-72pq4_29f1e4dd-119c-4a61-aaf1-8736217793fd/kube-rbac-proxy-web/0.log" Jul 02 22:19:51.021394 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:51.021313 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5854f8c9-72pq4_29f1e4dd-119c-4a61-aaf1-8736217793fd/kube-rbac-proxy/0.log" Jul 02 22:19:51.041740 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:51.041710 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5854f8c9-72pq4_29f1e4dd-119c-4a61-aaf1-8736217793fd/prom-label-proxy/0.log" Jul 02 22:19:51.063043 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:51.062996 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5854f8c9-72pq4_29f1e4dd-119c-4a61-aaf1-8736217793fd/kube-rbac-proxy-rules/0.log" Jul 02 22:19:51.083990 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:51.083887 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f5854f8c9-72pq4_29f1e4dd-119c-4a61-aaf1-8736217793fd/kube-rbac-proxy-metrics/0.log" Jul 02 22:19:52.334969 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:52.334938 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-w7bjn_701b2907-f252-40d2-a8f9-b22923946c39/networking-console-plugin/0.log" Jul 02 22:19:53.004862 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.004831 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/2.log" Jul 02 22:19:53.012363 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.012340 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-j6j5h_6e011139-283a-47f3-8490-1011cd239396/console-operator/3.log" Jul 02 22:19:53.560225 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.560195 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d6875dbb7-xpjxk_6f0babc3-5f42-4ebc-bd16-542d3e5c0429/console/0.log" Jul 02 22:19:53.599056 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.599021 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-ptjll_e65b4301-e5c4-4b64-b629-96dd9153001e/download-server/0.log" Jul 02 22:19:53.610255 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.610219 2547 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb"] Jul 02 22:19:53.617186 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.617145 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.626007 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.625970 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb"] Jul 02 22:19:53.715021 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.714989 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-sys\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.715226 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.715049 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vd9t\" (UniqueName: \"kubernetes.io/projected/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-kube-api-access-8vd9t\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.715226 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.715098 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-proc\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.715226 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.715122 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-lib-modules\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.715226 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.715166 2547 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-podres\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.815726 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.815643 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-sys\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.815726 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.815704 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vd9t\" (UniqueName: \"kubernetes.io/projected/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-kube-api-access-8vd9t\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.815952 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.815739 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-proc\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.815952 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.815764 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-lib-modules\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.815952 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.815770 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-sys\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.815952 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.815828 2547 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-podres\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.815952 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.815876 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-proc\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.816132 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.815978 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-lib-modules\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.816132 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.815982 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-podres\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.825231 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.825195 2547 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vd9t\" (UniqueName: \"kubernetes.io/projected/7b667cd7-c1d6-467e-b65f-1ccf57f8ced3-kube-api-access-8vd9t\") pod \"perf-node-gather-daemonset-lswlb\" (UID: \"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3\") " pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:53.936968 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:53.936933 2547 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:54.126354 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:54.126306 2547 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb"] Jul 02 22:19:54.128552 ip-10-0-137-35 kubenswrapper[2547]: W0702 22:19:54.128522 2547 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7b667cd7_c1d6_467e_b65f_1ccf57f8ced3.slice/crio-a4db1df9bc7d415ac23f9049acd114206a57a9d706dd786cdc5ea4fc32f33571 WatchSource:0}: Error finding container a4db1df9bc7d415ac23f9049acd114206a57a9d706dd786cdc5ea4fc32f33571: Status 404 returned error can't find the container with id a4db1df9bc7d415ac23f9049acd114206a57a9d706dd786cdc5ea4fc32f33571 Jul 02 22:19:54.272172 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:54.272110 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-lg2n6_b4ac7946-f208-4b2b-9462-3916a0a72e8e/volume-data-source-validator/0.log" Jul 02 22:19:54.508625 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:54.508531 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" event={"ID":"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3","Type":"ContainerStarted","Data":"183b7dbf73a479f8263714b7943b2404d24fd94bce4f466a3988eeb036f16ba3"} Jul 02 22:19:54.508800 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:54.508695 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:19:54.508800 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:54.508733 2547 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" event={"ID":"7b667cd7-c1d6-467e-b65f-1ccf57f8ced3","Type":"ContainerStarted","Data":"a4db1df9bc7d415ac23f9049acd114206a57a9d706dd786cdc5ea4fc32f33571"} Jul 02 22:19:54.525703 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:54.525646 2547 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" podStartSLOduration=1.525606128 podStartE2EDuration="1.525606128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 22:19:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 22:19:54.523946669 +0000 UTC m=+2928.973785297" watchObservedRunningTime="2026-07-02 22:19:54.525606128 +0000 UTC m=+2928.975444756" Jul 02 22:19:55.385045 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:55.385014 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvj7q_33485243-8457-4ab8-83a2-3ad6cc1d98f9/dns/0.log" Jul 02 22:19:55.403700 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:55.403661 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvj7q_33485243-8457-4ab8-83a2-3ad6cc1d98f9/kube-rbac-proxy/0.log" Jul 02 22:19:55.429631 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:55.429605 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l2nsz_25b068f3-c4ca-4d18-b74c-1b8d653d27c8/dns-node-resolver/0.log" Jul 02 22:19:56.052861 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:56.052823 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gjvt4_47e464a7-c6bb-45fa-89b9-a4c5eb701478/node-ca/0.log" Jul 02 22:19:56.974867 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:56.974837 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jjpt7_b3624094-3709-40d5-b87c-21645a759e14/istio-proxy/0.log" Jul 02 22:19:57.205035 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:57.205008 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d45cbf66b-c2f2f_4f9f105e-0883-4199-84ae-7c79c2f6f1dc/kube-auth-proxy/0.log" Jul 02 22:19:57.244710 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:57.244621 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-q7lbs_619e39a7-9426-48dc-a639-12103eeaead4/istio-proxy/0.log" Jul 02 22:19:57.308195 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:57.308155 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-q7lbs_619e39a7-9426-48dc-a639-12103eeaead4/istio-proxy/1.log" Jul 02 22:19:57.346711 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:57.346684 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8b9x9_5730229c-9c39-4ecf-84d8-2d4579577ca3/payload-pre-processing/0.log" Jul 02 22:19:57.428275 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:57.428237 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gwgmr_2a392b5e-9307-4bc5-971c-48848af0a307/payload-processing/0.log" Jul 02 22:19:57.451832 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:57.451800 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bc4c77f4f-2bbsx_5aa921ab-8003-4f10-83ea-9842fb74856a/router/0.log" Jul 02 22:19:58.125870 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:58.125831 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-grgfz_ac732629-79f4-4eea-95f4-f5e748d23c05/serve-healthcheck-canary/0.log" Jul 02 22:19:58.697200 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:58.697172 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2x644_6ca7ece0-b874-4113-88c2-11a0bf25a165/kube-rbac-proxy/0.log" Jul 02 22:19:58.721460 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:58.721419 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2x644_6ca7ece0-b874-4113-88c2-11a0bf25a165/exporter/0.log" Jul 02 22:19:58.750145 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:19:58.750115 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2x644_6ca7ece0-b874-4113-88c2-11a0bf25a165/extractor/0.log" Jul 02 22:20:00.532894 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:00.531372 2547 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-92hjf/perf-node-gather-daemonset-lswlb" Jul 02 22:20:01.047416 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.047350 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-jxqlb_69be0910-f104-433e-a97d-43ab24d5cd82/manager/0.log" Jul 02 22:20:01.301620 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.301528 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-7vbvm_ced64b90-2bad-4247-afd2-f4776cb690fd/manager/0.log" Jul 02 22:20:01.372236 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.372201 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717160-2b244_ea3b61bb-b6a8-4bad-a25a-031c89bab0a2/cleanup/0.log" Jul 02 22:20:01.397882 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.397851 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29717175-7kvmz_f26ebcd9-6645-444a-aea7-639d3b513f70/cleanup/0.log" Jul 02 22:20:01.575828 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.575741 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-794b6f76fd-ttmvw_1a074673-6489-4874-b9e5-723117d22873/manager/0.log" Jul 02 22:20:01.597435 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.597409 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-t577c_02d4eac5-d07a-4f24-893a-b24c5cec4b1d/server/0.log" Jul 02 22:20:01.711013 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.710965 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bqp65_49cca079-af7a-44a6-b044-574a00ec939f/manager/0.log" Jul 02 22:20:01.885081 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.884976 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4697655-gpt2x_85d72d11-b3d1-4c52-8556-bb51acdde7c5/manager/0.log" Jul 02 22:20:01.904302 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.904272 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c4697655-gpt2x_85d72d11-b3d1-4c52-8556-bb51acdde7c5/copy-manifests/0.log" Jul 02 22:20:01.987122 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:01.987094 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-88fd2_a548cb99-e8e8-4d1d-a196-35fbef85d57a/postgres/0.log" Jul 02 22:20:03.557948 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:03.557913 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-gvldl_688f550f-2a9e-41b0-a72d-770a2fc92847/openshift-lws-operator/0.log" Jul 02 22:20:03.655179 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:03.655145 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jjpt7_b3624094-3709-40d5-b87c-21645a759e14/istio-proxy/0.log" Jul 02 22:20:03.881543 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:03.881515 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d45cbf66b-c2f2f_4f9f105e-0883-4199-84ae-7c79c2f6f1dc/kube-auth-proxy/0.log" Jul 02 22:20:03.907296 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:03.907268 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-q7lbs_619e39a7-9426-48dc-a639-12103eeaead4/istio-proxy/0.log" Jul 02 22:20:03.965160 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:03.965127 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-q7lbs_619e39a7-9426-48dc-a639-12103eeaead4/istio-proxy/1.log" Jul 02 22:20:04.000513 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:04.000472 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8b9x9_5730229c-9c39-4ecf-84d8-2d4579577ca3/payload-pre-processing/0.log" Jul 02 22:20:04.075621 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:04.075590 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gwgmr_2a392b5e-9307-4bc5-971c-48848af0a307/payload-processing/0.log" Jul 02 22:20:04.092548 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:04.092518 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bc4c77f4f-2bbsx_5aa921ab-8003-4f10-83ea-9842fb74856a/router/0.log" Jul 02 22:20:05.334936 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:05.334846 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4jjpt7_b3624094-3709-40d5-b87c-21645a759e14/istio-proxy/0.log" Jul 02 22:20:05.544277 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:05.544249 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d45cbf66b-c2f2f_4f9f105e-0883-4199-84ae-7c79c2f6f1dc/kube-auth-proxy/0.log" Jul 02 22:20:05.566973 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:05.566932 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-q7lbs_619e39a7-9426-48dc-a639-12103eeaead4/istio-proxy/0.log" Jul 02 22:20:05.626585 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:05.626509 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-q7lbs_619e39a7-9426-48dc-a639-12103eeaead4/istio-proxy/1.log" Jul 02 22:20:05.657794 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:05.657757 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-8b9x9_5730229c-9c39-4ecf-84d8-2d4579577ca3/payload-pre-processing/0.log" Jul 02 22:20:05.725352 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:05.724918 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-gwgmr_2a392b5e-9307-4bc5-971c-48848af0a307/payload-processing/0.log" Jul 02 22:20:05.744026 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:05.743991 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7bc4c77f4f-2bbsx_5aa921ab-8003-4f10-83ea-9842fb74856a/router/0.log" Jul 02 22:20:10.315477 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.315449 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bswhr_b035aac5-e945-44dd-a4f6-131fca47ad48/kube-multus-additional-cni-plugins/0.log" Jul 02 22:20:10.334840 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.334812 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bswhr_b035aac5-e945-44dd-a4f6-131fca47ad48/egress-router-binary-copy/0.log" Jul 02 22:20:10.354603 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.354575 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bswhr_b035aac5-e945-44dd-a4f6-131fca47ad48/cni-plugins/0.log" Jul 02 22:20:10.373730 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.373698 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bswhr_b035aac5-e945-44dd-a4f6-131fca47ad48/bond-cni-plugin/0.log" Jul 02 22:20:10.394113 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.394088 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bswhr_b035aac5-e945-44dd-a4f6-131fca47ad48/routeoverride-cni/0.log" Jul 02 22:20:10.412694 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.412671 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bswhr_b035aac5-e945-44dd-a4f6-131fca47ad48/whereabouts-cni-bincopy/0.log" Jul 02 22:20:10.433085 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.433057 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bswhr_b035aac5-e945-44dd-a4f6-131fca47ad48/whereabouts-cni/0.log" Jul 02 22:20:10.514642 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.514582 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-szhvf_d20d9255-7e0d-4fd4-a553-18bb3769366c/kube-multus/0.log" Jul 02 22:20:10.611904 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.611811 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-95lhp_f4f1ee0d-0554-4134-b502-08c8aefca7ec/network-metrics-daemon/0.log" Jul 02 22:20:10.626487 ip-10-0-137-35 kubenswrapper[2547]: I0702 22:20:10.626462 2547 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-95lhp_f4f1ee0d-0554-4134-b502-08c8aefca7ec/kube-rbac-proxy/0.log"